Showing posts with label puppet. Show all posts
Showing posts with label puppet. Show all posts
Friday, March 27, 2015
Clash of Titans, Ansible vs Puppet vs Salt
I have read this excellent article
https://dantehranian.wordpress.com/2015/01/20/ansible-vs-puppet-overview/
with part 2 https://dantehranian.wordpress.com/2015/01/20/ansible-vs-puppet-hands-on-with-ansible/
I have been dreaming of writing the same article for a long time....
2 statements caught my attention
"“I would say that if you don’t have a Puppet champion (someone that knows it well and is pushing to have it running) it is probably not worth for this.""
" I tend to agree with Lyft’s conclusion that if you have a centralized Ops team in change of deployments then they can own a Puppet codebase. On the other hand if you want more wide-spread ownership of your configuration management scripts, a tool with a shallower learning curve like Ansible is a better choice."
I haven't used Ansible, but such is my experience with Puppet: you need a technical guru (or max 2) dictating standards and finding solutions to common problems and bridging communication among departments and teams.... you can't just "leave it to the masses" and hoping everybody will be a champion... also, who wants Ruby? All my stuff is in Python !!!
And here goes a very interesting comparison Ansible vs Salt...
And here a "let's move away from Puppet, either to Salt or to Ansible" post
Thursday, March 5, 2015
Installing Weblogic 11.1.6 with Java 7 in a few seconds
(maybe the few seconds is an overstatement....)
With the Biemond way (thank you Biemond you are a God):
install git and vagrant
Download jdk-7u55-linux-x64.tar.gz from http://www.oracle.com/technetwork/java/javase/downloads/java-archive-downloads-javase7-521261.html#jdk-7u55-oth-JPR
Download UnlimitedJCEPolicyJDK7.zip (JCE Java Cryptographic Extensions) here http://www.oracle.com/technetwork/java/javase/downloads/jce-7-download-432124.html
Download wls1036_generic.jar from http://www.oracle.com/technetwork/middleware/ias/downloads/wls-main-097127.html (choose "Oracle WebLogic Server 11gR1 (10.3.6) + Coherence - Package Installer" for "Additional Platforms", where it says "File1 1 GB" )
Download from https://support.oracle.com the p17572726_1036_Generic.zip "Patch 17572726: SU Patch [FCX7]: WLS PATCH SET UPDATE 10.3.6.0.7" https://support.oracle.com/epmos/faces/PatchDetail?requestId=16925196&_afrLoop=481943043189464&patchId=17572726&_afrWindowMode=0&_adf.ctrl-state=jhdx9weyr_233
Checkout the Biemond Vagrant from here https://github.com/biemond/biemond-orawls-vagrant (copy the http url from the "HTTPS clone URL" box)
git clone https://github.com/biemond/biemond-orawls-vagrant.git
put the downloaded jars and zips and tar.gz into /Users/edwin/software
vagrant up
It should all work. If not, use your brain.
If you don't want to use Biemond's Puppet module, but rather waste countless hours doing it manually, you can download a CentOS Appliance (OVA file)
http://virtualboxes.org/images/centos/
download https://s3-eu-west-1.amazonaws.com/virtualboxes.org/CentOS7-base.ova.torrent (you can download Free Download Manager to download it)
Open VirtualBox, Import appliance, select the .ova file just downloaded
See also
http://www.javamonamour.org/2014/09/weblogic-installing-weblogic-1035.html
With the Biemond way (thank you Biemond you are a God):
install git and vagrant
Download jdk-7u55-linux-x64.tar.gz from http://www.oracle.com/technetwork/java/javase/downloads/java-archive-downloads-javase7-521261.html#jdk-7u55-oth-JPR
Download UnlimitedJCEPolicyJDK7.zip (JCE Java Cryptographic Extensions) here http://www.oracle.com/technetwork/java/javase/downloads/jce-7-download-432124.html
Download wls1036_generic.jar from http://www.oracle.com/technetwork/middleware/ias/downloads/wls-main-097127.html (choose "Oracle WebLogic Server 11gR1 (10.3.6) + Coherence - Package Installer" for "Additional Platforms", where it says "File1 1 GB" )
Download from https://support.oracle.com the p17572726_1036_Generic.zip "Patch 17572726: SU Patch [FCX7]: WLS PATCH SET UPDATE 10.3.6.0.7" https://support.oracle.com/epmos/faces/PatchDetail?requestId=16925196&_afrLoop=481943043189464&patchId=17572726&_afrWindowMode=0&_adf.ctrl-state=jhdx9weyr_233
Checkout the Biemond Vagrant from here https://github.com/biemond/biemond-orawls-vagrant (copy the http url from the "HTTPS clone URL" box)
git clone https://github.com/biemond/biemond-orawls-vagrant.git
put the downloaded jars and zips and tar.gz into /Users/edwin/software
vagrant up
It should all work. If not, use your brain.
If you don't want to use Biemond's Puppet module, but rather waste countless hours doing it manually, you can download a CentOS Appliance (OVA file)
http://virtualboxes.org/images/centos/
download https://s3-eu-west-1.amazonaws.com/virtualboxes.org/CentOS7-base.ova.torrent (you can download Free Download Manager to download it)
Open VirtualBox, Import appliance, select the .ova file just downloaded
See also
http://www.javamonamour.org/2014/09/weblogic-installing-weblogic-1035.html
Monday, November 10, 2014
Install puppet modules recursively from a Puppetfile
If you have worked with r10k or puppet-librarian (or even worse, if you have managed Puppet modules manually) you must have realized how pathetic these tools are.
Luckily now Bruno Bieth comes to our rescue with a brilliant Swiss (actually: French!) knife:
https://github.com/backuity/puppet-module-installer
The tool is VERY simple to install (no gem, no ruby, no crap) and it gives you invaluable dependency analysis, consistency checks and graphs that will unveil any dodgy situation in your Puppet module dependencies.
Luckily now Bruno Bieth comes to our rescue with a brilliant Swiss (actually: French!) knife:
https://github.com/backuity/puppet-module-installer
The tool is VERY simple to install (no gem, no ruby, no crap) and it gives you invaluable dependency analysis, consistency checks and graphs that will unveil any dodgy situation in your Puppet module dependencies.
Labels:
puppet
Monday, September 29, 2014
Puppet: file recurse remote caveat
Puppet contains a beautiful "macro" file recurse remote, by which you keep a while directory (with subdirectories!) in sync with the content of a Puppet files repository. Great.
BUT. There is a BUT: if apart from maintaining the whole folder (say: /myfiles) with recurse-remote, you also maintain a file in a subfolder (say: /myfiles/myfolder/hello.txt), this breaks the whole synchronization of /myfiles/myfolder/. OTHER subfolders (say: /myfiles/myotherfolder/) will syncronize perfectly, but not /myfiles/myfolder/.
Now you know, so you can plan your workarounds to this - maybe unexpected - behavior.
BUT. There is a BUT: if apart from maintaining the whole folder (say: /myfiles) with recurse-remote, you also maintain a file in a subfolder (say: /myfiles/myfolder/hello.txt), this breaks the whole synchronization of /myfiles/myfolder/. OTHER subfolders (say: /myfiles/myotherfolder/) will syncronize perfectly, but not /myfiles/myfolder/.
Now you know, so you can plan your workarounds to this - maybe unexpected - behavior.
Labels:
puppet
Tuesday, September 23, 2014
Puppet: mount a SAMBA share at boot
First, create a credentials file in your acme module:
acme\files\samba\myshare.smb.credentials
and inside put:
I was unable to make it mount an unprotected share..... so just have it protected by username/password.
Then in your acme init.pp read a boolean
$acme_install_sambamyshare = any2bool(hiera('acme_install_sambamyshare', false)),
Create a samba.pp class (file):
Don't forget to declare the samba class in your init.pp!
If you do cat /etc/fstab you should see something like this:
//windowshost/sharedfolder /data/myshare/ cifs credentials=/etc/myshare.smb.credentials,rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777 0 0
If you want to do the same thing manually, this is the recipe:
yum install smbclient
//yum install cifs-utils for Ubuntu
/bin/mount -t cifs -o username=someuser,password=somepassword,rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777 //windowshost/sharedfolder /data/myshare/
To remove a mountpoint, use umount:
umount /data/myshare/
it supports also a -f (force) option:
umount -f /data/myshare/
See also here
acme\files\samba\myshare.smb.credentials
and inside put:
username=myuser password=mypassword
I was unable to make it mount an unprotected share..... so just have it protected by username/password.
Then in your acme init.pp read a boolean
$acme_install_sambamyshare = any2bool(hiera('acme_install_sambamyshare', false)),
Create a samba.pp class (file):
# == Class: acme::samba
#
# Manages samba mounts
#
class acme::samba {
if $acme::acme_install_sambamyshare {
file { '/data/myshare/':
ensure => directory,
owner => "soa",
group => 'soa',
mode => '0775',
}
file { '/etc/myshare.smb.credentials': source => 'puppet:///modules/acme/samba/myshare.smb.credentials',
owner => root,
mode => '0700', } ->
mount { 'myshare':
name => '/data/myshare/',
atboot => 'true',
device => '//windowshost/sharedfolder',
ensure => 'mounted',
fstype => 'cifs',
options => "credentials=/etc/myshare.smb.credentials,rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777",
require => [Package['samba-client'], Package['cifs-utils'], File['/data/myshare/'], File['/etc/myshare.smb.credentials']],
}
}
}
Don't forget to declare the samba class in your init.pp!
If you do cat /etc/fstab you should see something like this:
//windowshost/sharedfolder /data/myshare/ cifs credentials=/etc/myshare.smb.credentials,rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777 0 0
If you want to do the same thing manually, this is the recipe:
yum install smbclient
//yum install cifs-utils for Ubuntu
/bin/mount -t cifs -o username=someuser,password=somepassword,rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777 //windowshost/sharedfolder /data/myshare/
To remove a mountpoint, use umount:
umount /data/myshare/
it supports also a -f (force) option:
umount -f /data/myshare/
See also here
Monday, September 15, 2014
Puppet: quick hiera setup
cat /home/soa/.puppet/hiera.yaml
cat /home/soa/common.yaml
To test if hiera is working:
this should print:
:hierarchy:
- common
:backends:
- yaml
:yaml:
:datadir: '/home/soa'
cat /home/soa/common.yaml
install_cleanssbatchorder : true
To test if hiera is working:
puppet apply -e "if hiera('install_cleanssbatchorder', false) { notify { 'pippo':}} "
this should print:
notice: pippo notice: /Stage[main]//Notify[pippo]/message: defined 'message' as 'pippo' notice: Finished catalog run in 0.02 seconds
Sunday, September 14, 2014
Puppet: minimalistic custom fact
in your module, create a lib/facter folder.
There, create a customxpathversion.rb file containing this code:
where the file customxpathversion.txt contains "1.10"
Now if in your puppet code you do
you get 1.10
Of course one should handle errors etc. But it works and I don't have to bother about $LOAD_PATH. From command line "facter customxpathversion" will not work because probably it's not in the LOAD_PATH.
Here the whole documentation Update: you can use this custom fact from the facter command line if you set the FACTERLIB environment variable:
There, create a customxpathversion.rb file containing this code:
require 'puppet'
Facter.add("customxpathversion") do
setcode 'cat /opt/oracle/scripts/config/customxpathversion.txt'
end
where the file customxpathversion.txt contains "1.10"
Now if in your puppet code you do
notify { "${::customxpathversion}" : }
you get 1.10
Of course one should handle errors etc. But it works and I don't have to bother about $LOAD_PATH. From command line "facter customxpathversion" will not work because probably it's not in the LOAD_PATH.
Here the whole documentation Update: you can use this custom fact from the facter command line if you set the FACTERLIB environment variable:
find / -name customxpathversion* /tmp/vagrant-puppet-3/modules-0/pippo/lib/facter/customxpathversion.rb ^C [soa@osb-vagrant ~]$ export FACTERLIB=/tmp/vagrant-puppet-3/modules-0/pippo/lib/facter/ [soa@osb-vagrant ~]$ facter customxpathversion 1.10
Labels:
puppet
Monday, June 30, 2014
Book: Extending Puppet
Alessandro "the Great" Franceschi, prominent Puppet guru, just authored this EXCELLENT book...
Not only the book is written in a fresh and speedy language, but it really covers ALL the big important topics related to managing a complex Puppet infrastructure, using the latest technology available... PuppetDB, Foreman, Hiera with Puppet 3...
However.... I am not a big fan of Puppet, I find it really outdated in its core design and choice to develop an independent DSL rather than relying on some existing language... but one thing that REALLY strikes me weird is how late the Puppet galaxy started worrying about separating configuration from code... something that in the Java world has been done since the origin of the language, with property files with all formats, databases, any pluggable datasource you can imagine... in Puppet they started worrying only in 2010, and still struggling to define standards...Infrastructure as code, wonderful, but at least give me the power of Java 8... AT LEAST ! I am not asking for Scala...
Saturday, June 21, 2014
Book: creating development environments with vagrant
http://www.packtpub.com/creating-development-environments-with-vagrant/book
This is a really cool, down to earth, getting started tutorial guiding you through some simple configurations with Vagrant and Puppet/Chef.
A few recipes are shown to
- create your own Virtual Box
- configure the network
- install apache, mysql etc etc
One should really go through this book when getting started with Vagrant and Puppet, great value for the money.
This is a really cool, down to earth, getting started tutorial guiding you through some simple configurations with Vagrant and Puppet/Chef.
A few recipes are shown to
- create your own Virtual Box
- configure the network
- install apache, mysql etc etc
One should really go through this book when getting started with Vagrant and Puppet, great value for the money.
Monday, June 2, 2014
Weird Hiera related error messages in Puppet
A colleague had a YAML file not properly formatted, and he kept getting this error message:
Could not retrieve catalog from remote server: Error 400 on SERVER: syntax error on line 16, col -1: `' at /etc/puppetlabs/puppet/environments/dev/modules/introscope/manifests/pippo.pp:22 on node mynode.acme.com
at line pippo.pp:22 there is a call to hiera
One would say that hiera is so poorly coded that it doesn't even bother to log that there is a YAML parsing error... I remember working in GWBasic 30 years ago and most errors were reported in the same way, "syntax error"... but it was 30 years ago.... in fact working with Puppet I have often this 1980 feeling...
Could not retrieve catalog from remote server: Error 400 on SERVER: syntax error on line 16, col -1: `' at /etc/puppetlabs/puppet/environments/dev/modules/introscope/manifests/pippo.pp:22 on node mynode.acme.com
at line pippo.pp:22 there is a call to hiera
One would say that hiera is so poorly coded that it doesn't even bother to log that there is a YAML parsing error... I remember working in GWBasic 30 years ago and most errors were reported in the same way, "syntax error"... but it was 30 years ago.... in fact working with Puppet I have often this 1980 feeling...
Tuesday, April 8, 2014
puppet create directory recursively
The whole VERY OLD rant is here http://projects.puppetlabs.com/issues/86
Why on earth in 8 years the Puppet Gods have kept ignoring this very basic and common need, they only know.
In a nutshell, I have a variable
$mypath='/opt/oracle/fmw11_1_1_6'
file { "$mypath":
ensure => directory,
owner => 'soa'
}
This will of course fail if /opt and /opt/oracle have not been already defined.
Of course since $mypath is a configuration value there is no way I can hardcode all the parents directories.
Hence I am screwed, and Puppet doesn't offer me any way around.
I can think of only one elegant way, that is writing a Ruby function (I am not good in Puppet stinky DSL) "pathExpand" to transform a full path into an array of his components:
assert [ '/opt', '/opt/oracle', '/opt/oracle/fmw11_1_1_6' ] = pathExpand('/opt/oracle/fmw11_1_1_6')
and a define "conditionalDirectory($mypath ):
if (! defined($mypath)) {
file { "$mypath":
ensure => directory,
}
}
All this sucks, but not as much as Puppet DSL anyway.
This is a Puppet solution (hack) to the problem:
Why on earth in 8 years the Puppet Gods have kept ignoring this very basic and common need, they only know.
In a nutshell, I have a variable
$mypath='/opt/oracle/fmw11_1_1_6'
file { "$mypath":
ensure => directory,
owner => 'soa'
}
This will of course fail if /opt and /opt/oracle have not been already defined.
Of course since $mypath is a configuration value there is no way I can hardcode all the parents directories.
Hence I am screwed, and Puppet doesn't offer me any way around.
I can think of only one elegant way, that is writing a Ruby function (I am not good in Puppet stinky DSL) "pathExpand" to transform a full path into an array of his components:
assert [ '/opt', '/opt/oracle', '/opt/oracle/fmw11_1_1_6' ] = pathExpand('/opt/oracle/fmw11_1_1_6')
and a define "conditionalDirectory($mypath ):
if (! defined($mypath)) {
file { "$mypath":
ensure => directory,
}
}
All this sucks, but not as much as Puppet DSL anyway.
This is a Puppet solution (hack) to the problem:
define common::mkdirp(
$owner = 'root',
$group = 'root',
$exec_path = '/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin') {
exec { "create ${name}":
command => "mkdir -p ${name}",
creates => $name,
path => $exec_path,
}
common::ownership { $name:
user => $owner,
group => $group,
exec_path => $exec_path,
require => Exec["create ${name}"]
}
}
define common::ownership(
$user,
$group = undef,
$dir = $name,
$exec_path = '/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin'
) {
$real_group = $group ? {
undef => $user,
default => $group
}
exec { "full '${dir}' directory ownership for ${user}:${real_group}":
path => $exec_path,
command => "chown -R ${user}:${real_group} ${dir}",
unless => "test `find ${dir} -user ${user} -group ${real_group} | wc -l` -eq `find ${dir} | wc -l`",
}
}
Labels:
puppet
Friday, April 4, 2014
Object Oriented Puppet
One of the most shocking aspects of working with Puppet is that it goes to a great length at providing a High Level Interface to Resources, but it provides literally no facility to abstract access to configuration values.
All is a hiera() and Hash/Array manipulation, from which to painfully extract subset of data with totally garbled and complex manipulations of the Hashes, Keys and Values, interspersed with some create_resources() to be able to cycle over a collection of homogeneous resources.
People who come from ANY Object Oriented, functional and fluent-interface programming languages, like Groovy or Java 8, can only scream in horror in front of the total inelegance, un-refactorability, un-maintainability and lack of encapsulation of the Puppet DSL.
To this, add that YAML is not generally associated to a schema - hence you can't validate that you haven't broken your model somewhere in your hundred of YAML files - on per each host - and you have the mother of all Spaghetti Western Code.
For all these reasons, please Puppet Gods, heed my call, dump that pathetic DSL and adopt some OO language.
All is a hiera() and Hash/Array manipulation, from which to painfully extract subset of data with totally garbled and complex manipulations of the Hashes, Keys and Values, interspersed with some create_resources() to be able to cycle over a collection of homogeneous resources.
People who come from ANY Object Oriented, functional and fluent-interface programming languages, like Groovy or Java 8, can only scream in horror in front of the total inelegance, un-refactorability, un-maintainability and lack of encapsulation of the Puppet DSL.
To this, add that YAML is not generally associated to a schema - hence you can't validate that you haven't broken your model somewhere in your hundred of YAML files - on per each host - and you have the mother of all Spaghetti Western Code.
For all these reasons, please Puppet Gods, heed my call, dump that pathetic DSL and adopt some OO language.
Labels:
puppet
Monday, March 24, 2014
in Puppet (and Linux) every slash counts
I was creating a symbolic link:
on a machine where such a link was already defined, but without the trailing / ( /usr/lib/jvm/java-1.6.0-sun.x86_64 ).
Much to my surprise, Puppet acted on that link by changing it to /usr/lib/jvm/java-1.6.0-sun.x86_64/ (trailing /) ... as there were any difference between the two.
Funnily, Linux itself shows a different entry after the change:
before:
lrwxrwxrwx 1 soa soa 34 Mar 24 18:27 java -> /usr/lib/jvm/java-1.6.0-sun.x86_64
after
lrwxrwxrwx 1 soa soa 34 Mar 24 18:27 java -> /usr/lib/jvm/java-1.6.0-sun.x86_64/
Well.... I think WWII was worse...
file { "/opt/oracle/java/" :
ensure => link,
target => "/usr/lib/jvm/java-1.6.0-sun.x86_64/",
}
on a machine where such a link was already defined, but without the trailing / ( /usr/lib/jvm/java-1.6.0-sun.x86_64 ).
Much to my surprise, Puppet acted on that link by changing it to /usr/lib/jvm/java-1.6.0-sun.x86_64/ (trailing /) ... as there were any difference between the two.
Funnily, Linux itself shows a different entry after the change:
before:
lrwxrwxrwx 1 soa soa 34 Mar 24 18:27 java -> /usr/lib/jvm/java-1.6.0-sun.x86_64
after
lrwxrwxrwx 1 soa soa 34 Mar 24 18:27 java -> /usr/lib/jvm/java-1.6.0-sun.x86_64/
Well.... I think WWII was worse...
Puppet file recurse
to purge or not to purge?
to recurse or not to recurse?
I have a bunch of files in a mymodule/files/myfiles folder inside the puppet module mymodule. I need to copy them to a target folder. I don't want to specify the INDIVIDUAL files, so I want to use the "recurse" option.
This will do absolutely NOTHING:
This will change mode to ALL files under /opt/oracle/fmw11_1_1_5/wlserver_10.3/common/wlst/modules/ (including subdirectories), but will NOT remove them:
This will remove any file which is not in the source, but subdirectories will remain:
This is the least-impacting: it will not change existing files, and will just copy the files in source:
to recurse or not to recurse?
I have a bunch of files in a mymodule/files/myfiles folder inside the puppet module mymodule. I need to copy them to a target folder. I don't want to specify the INDIVIDUAL files, so I want to use the "recurse" option.
This will do absolutely NOTHING:
file { '/opt/oracle/fmw11_1_1_5/wlserver_10.3/common/wlst/modules/' :
source => "puppet:///modules/mymodule/myfiles/",
}
This will change mode to ALL files under /opt/oracle/fmw11_1_1_5/wlserver_10.3/common/wlst/modules/ (including subdirectories), but will NOT remove them:
file { '/opt/oracle/fmw11_1_1_5/wlserver_10.3/common/wlst/modules/' :
source => "puppet:///modules/mymodule/myfiles/",
recurse => true
}
This will remove any file which is not in the source, but subdirectories will remain:
file { '/opt/oracle/fmw11_1_1_5/wlserver_10.3/common/wlst/modules/' :
source => "puppet:///modules/mymodule/myfiles/",
recurse => true,
purge => true,
}
This is the least-impacting: it will not change existing files, and will just copy the files in source:
file { '/opt/oracle/fmw11_1_1_5/wlserver_10.3/common/wlst/modules/' :
source => "puppet:///modules/mymodule/myfiles/",
recurse => remote
}
Labels:
puppet
Thursday, March 20, 2014
My first custom Puppet function in Ruby
Here the doc http://docs.puppetlabs.com/guides/custom_functions.html
I thought this was Rocket Science, but it's quite simple, once you overcome the normal, healthy, instinctive repugnance for a stinky stupid language like Ruby. Anyway if you are already coping with Puppet, Ruby should not be worse.
Inside your Puppet module, create these folders: lib/puppet/parser/functions, and inside your function file "stage_from_domain.rb" (file name should match the function name, and extension = .rb).
The file should start with a "module Puppet::Parser::Functions" declaration (no clue why... just to make your pronounce some meaningless magic words initiating you to the Puppet Mysteries - luckily they don't ask you to kill a kitten).
The function returns a substring in the middle of a string. Since we RETURN a value, we must declare it as ":type => :rvalue":
That's all, you can now use the stage_from_domain function inside Puppet. Cool! Maybe after all I will be able to turn Puppet into something manageable... especially the day they will decide to support Groovy, instead of Stinking Ruby.
I thought this was Rocket Science, but it's quite simple, once you overcome the normal, healthy, instinctive repugnance for a stinky stupid language like Ruby. Anyway if you are already coping with Puppet, Ruby should not be worse.
Inside your Puppet module, create these folders: lib/puppet/parser/functions, and inside your function file "stage_from_domain.rb" (file name should match the function name, and extension = .rb).
The file should start with a "module Puppet::Parser::Functions" declaration (no clue why... just to make your pronounce some meaningless magic words initiating you to the Puppet Mysteries - luckily they don't ask you to kill a kitten).
The function returns a substring in the middle of a string. Since we RETURN a value, we must declare it as ":type => :rvalue":
module Puppet::Parser::Functions
newfunction(:stage_from_domain, :type => :rvalue) do |args|
#e.g. domain_name='osbpr1do'
domain_name = args[0]
#we should remove the leading "osb" and the trailing "do"
if domain_name[0, 3] != 'osb'
raise ArgumentError, 'domain_name should start with osb'
end
if domain_name[domain_name.length - 2, domain_name.length] != 'do'
raise ArgumentError, 'domain_name should end with do'
end
domain_name[3, domain_name.length - 5]
end
end
The REALLY weirdo thing is how to return the value: simply end your function specifying the rvalue to return (in my case "domain_name[3, domain_name.length - 5]". No old-fashioned, READABLE "return" statements... who needs clarity....
That's all, you can now use the stage_from_domain function inside Puppet. Cool! Maybe after all I will be able to turn Puppet into something manageable... especially the day they will decide to support Groovy, instead of Stinking Ruby.
Monday, March 17, 2014
Puppet. generate a property file from a YAML hash
the YAML:
The file/template statenent:
The acmeconfig.properties.erb template file:
The generated property file:
acmev2_properties: acmev2_env : DEV acmev2_loadbalancerurl : http://hqchacme102.acme.com:8001/
The file/template statenent:
file { "${acmescripts_rootfolder}config/acmeconfig.properties":
ensure => present,
content => template('acmev2/acmescripts/acmeconfig.properties.erb'),
mode => 0775,
}
The acmeconfig.properties.erb template file:
<% @acmev2_properties.each do |key, value| -%> <%= key %>=<%= value %> <% end -%>
The generated property file:
acmev2_env=DEV acmev2_loadbalancerurl=http://hqchacme102.acme.com:8001/
Labels:
puppet
Thursday, March 6, 2014
puppet file recurse purge filebucket
Using
info: /File[/path/to/target/bireports/DailyReport_2014-3-5.txt]: Filebucketed /path/to/target/bireports/DailyReport_2014-3-5.txt to main with sum 8f629af878d7d7fd7b70d33502100a76
notice: /File[/path/to/target/bireports/DailyReport_2014-3-5.txt]/ensure: removed
So the message is: be very careful to use "purge" if the target folder is supposed to contain plenty of other stuff. Consider moving that stuff under another folder. Or just forget the "purge".
See also:
http://docs.puppetlabs.com/references/latest/type.html#file-attribute-recurse
Incidentally in my case there were so many files that it ended up with a "Error 400 on SERVER: Could not intern from pson: regexp buffer overflow"
file { "/path/to/target/":
source => "/path/to/staging/",
recurse => true,
purge => true,
}
has one big caveat: if /path/to/target/ contains a lot of uncontrolled stuff which doesn't exist in /path/to/staging/, all this redundant stuff will be deleted and filebucketed (presumably across the network to a main file bucket running on the puppet master), and you will see this message:
info: /File[/path/to/target/bireports/DailyReport_2014-3-5.txt]: Filebucketed /path/to/target/bireports/DailyReport_2014-3-5.txt to main with sum 8f629af878d7d7fd7b70d33502100a76
notice: /File[/path/to/target/bireports/DailyReport_2014-3-5.txt]/ensure: removed
So the message is: be very careful to use "purge" if the target folder is supposed to contain plenty of other stuff. Consider moving that stuff under another folder. Or just forget the "purge".
See also:
http://docs.puppetlabs.com/references/latest/type.html#file-attribute-recurse
Incidentally in my case there were so many files that it ended up with a "Error 400 on SERVER: Could not intern from pson: regexp buffer overflow"
Labels:
puppet
Friday, February 14, 2014
managing cron entries with Puppet
Puppet has a predefined type cron. Which is very cool. The only issue is that we don't want to hardcode cron stuff in any pp manifest, since each node has to be configured in a different way.
This is where hiera comes in handy. I personally dislike hiera, and in general all repositories of information which are not bound to a tightly validating schema and which don't natively lend themselves to being queried with a query language. Anyway this is another story.
This is the Puppet class:
and in your <hostname>.yaml file enter:
I can't think of anything simpler.
NOTA BENE:
the empty hash {} is needed so that, if a <hostname>.yaml doesn't define the acme_crontab_entries variable, you don't get the message
Could not find data item acme_crontab_entries in any Hiera data file and no default supplied at /tmp/vagrant-puppet/modules-0/acme/manifests/crontab.pp:6 on node osb-vagrant.acme.com
However this is a bit of abuse of hiera.... hiera should contain minimal configuration information, letting the Puppet modules to handle the details.
A more structured approach is to define in hiera only a boolean flag determining is a given cron entry is needed on a specific server: "acme_install_logrotate_service : true" , and then in your init.pp you do:
This is where hiera comes in handy. I personally dislike hiera, and in general all repositories of information which are not bound to a tightly validating schema and which don't natively lend themselves to being queried with a query language. Anyway this is another story.
This is the Puppet class:
class acme::crontab ($acme_crontab_entries = hiera('acme_crontab_entries', {})) {
create_resources(cron, $acme_crontab_entries)
}
and in your <hostname>.yaml file enter:
acme_crontab_entries :
logrotate:
command : '/usr/sbin/logrotate'
user : 'soa'
hour : '2'
minute : '0'
I can't think of anything simpler.
NOTA BENE:
hiera('acme_crontab_entries', {})
the empty hash {} is needed so that, if a <hostname>.yaml doesn't define the acme_crontab_entries variable, you don't get the message
Could not find data item acme_crontab_entries in any Hiera data file and no default supplied at /tmp/vagrant-puppet/modules-0/acme/manifests/crontab.pp:6 on node osb-vagrant.acme.com
However this is a bit of abuse of hiera.... hiera should contain minimal configuration information, letting the Puppet modules to handle the details.
A more structured approach is to define in hiera only a boolean flag determining is a given cron entry is needed on a specific server: "acme_install_logrotate_service : true" , and then in your init.pp you do:
$acme_install_logrotate_service = hiera('acme_install_logrotate_service', false)
if $acme_install_logrotate_service {
class {'acme::logrotateservices': }
}
and the class logrotateservices contains the Puppet statement:
cron { logrotate:
command => "/usr/sbin/logrotate",
user => root,
hour => ['2-4'],
minute => '*/10'
}
Labels:
puppet
Monday, December 2, 2013
Puppet and WebLogic the Biemond way
I always follow Biemond post, on the the richest sources of inspiration for all Oracle Professionals.
Here he makes his WebLogic/Puppet module available to the public:
http://biemond.blogspot.ch/2013/11/new-puppet-3-weblogic-provisioning.html
I login as root on my vagrant VirtualBox (RHEL) and I type:
puppet --version
3.2.4 (Puppet Enterprise 3.0.1)
Then I type
puppet module install biemond/orawls
I see that he has also provided a Vagrant box https://github.com/biemond/biemond-orawls-vagrant. I do a "git clone https://github.com/biemond/biemond-orawls-vagrant" and I got the biemond-orawls-vagrant folder with my vagrantfile in it. I cd biemond-orawls-vagrant and do vagrant up.
I get this nasty message
I do vagrant --version
Vagrant version 1.1.5
Ok let's download the latest 1.3.5 vagrant .
and install it (it takes a looooong time, reboot required )
I repeat "vagrant up" and this time Vagrant is happy and it downloads the centos-6.4-x86_64 base box (600 MB, it can take a long time depending on the connection speed)
The second attempt fails because there is no /vagrant/jdk-7u45-linux-x64.tar.gz file
I download the missing file from http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html and I put the jdk-7u45-linux-x64.tar.gz file in the same folder where I keep the vagrantfile (it will be synced by Vagrant, and mapped to /vagrant/jdk-7u45-linux-x64.tar.gz - after a vagrant destroy and another vagrant up... ).
When you connect to the VBOX, configure your putty with 127.0.0.1 port 2222 (note that in the VirtualBOx, it's also specified the IP 127.0.0.1, otherwise each time the IP of the host changes, you have to change the putty configuration...)
This time around, it complains that /data/install/wls1036_generic.jar is not found.
I download it from here http://download.oracle.com/otn/nt/middleware/11g/wls/1036/wls1036_generic.jar and manually copy it to /data/install and to the /vagrant share (melius abundare quam deficere) in the VBOX
I run "vagrant provision".
Again, p17071663_1036_Generic.zip is missing (ah if only I had read the doc before...)...
to get this file, you must log into support.oracle.com, click on "patches", specify the WebLogic 10.3.6 Product for Any platform and you will be presented with only one download: "SU Patch [BYJ1]: WLS PATCH SET UPDATE 10.3.6.0.6 (Patch) p17071663_1036_Generic.zip"
This time around I get a "Error: /Stage[main]/Bsu/Orawls::Bsu[BYJ1]/Exec[exec bsu ux BYJ1]/returns: change from notrun to 0 failed: Command exceeded timeout" after the extraction of the patchset.
For the OSB version, refer to:
https://github.com/matthewbaldwin/vagrant-osb
which is really well documented.
PS Important: don't run "vagrant up" because the node1 will copydomain from admin node BEFORE this is ready. You should first do "vagrant up admin", then "vagrant up node1" and vagrant up node2". After the "vagrant up admin" the admin server is running and accessble from the host machine with http://10.10.10.10:7001/console/ (this is not a global ip, it will work only locally)
Here he makes his WebLogic/Puppet module available to the public:
http://biemond.blogspot.ch/2013/11/new-puppet-3-weblogic-provisioning.html
I login as root on my vagrant VirtualBox (RHEL) and I type:
puppet --version
3.2.4 (Puppet Enterprise 3.0.1)
Then I type
puppet module install biemond/orawls
Notice: Preparing to install into /etc/puppetlabs/puppet/modules ... Notice: Created target directory /etc/puppetlabs/puppet/modules Notice: Downloading from https://forge.puppetlabs.com ... Notice: Installing -- do not interrupt ... /etc/puppetlabs/puppet/modules âââ biemond-orawls (v0.2.1)Cool!
I see that he has also provided a Vagrant box https://github.com/biemond/biemond-orawls-vagrant. I do a "git clone https://github.com/biemond/biemond-orawls-vagrant" and I got the biemond-orawls-vagrant folder with my vagrantfile in it. I cd biemond-orawls-vagrant and do vagrant up.
I get this nasty message
Vagrant has detected that you have a version of VirtualBox installed that is not supported. Please install one of the supported versions listed below to use Vagrant: 4.0, 4.1, 4.2and in fact I have the latest 4.3.4 version of VirtualBox.
I do vagrant --version
Vagrant version 1.1.5
Ok let's download the latest 1.3.5 vagrant .
and install it (it takes a looooong time, reboot required )
I repeat "vagrant up" and this time Vagrant is happy and it downloads the centos-6.4-x86_64 base box (600 MB, it can take a long time depending on the connection speed)
The second attempt fails because there is no /vagrant/jdk-7u45-linux-x64.tar.gz file
I download the missing file from http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html and I put the jdk-7u45-linux-x64.tar.gz file in the same folder where I keep the vagrantfile (it will be synced by Vagrant, and mapped to /vagrant/jdk-7u45-linux-x64.tar.gz - after a vagrant destroy and another vagrant up... ).
When you connect to the VBOX, configure your putty with 127.0.0.1 port 2222 (note that in the VirtualBOx, it's also specified the IP 127.0.0.1, otherwise each time the IP of the host changes, you have to change the putty configuration...)
This time around, it complains that /data/install/wls1036_generic.jar is not found.
I download it from here http://download.oracle.com/otn/nt/middleware/11g/wls/1036/wls1036_generic.jar and manually copy it to /data/install and to the /vagrant share (melius abundare quam deficere) in the VBOX
I run "vagrant provision".
Again, p17071663_1036_Generic.zip is missing (ah if only I had read the doc before...)...
to get this file, you must log into support.oracle.com, click on "patches", specify the WebLogic 10.3.6 Product for Any platform and you will be presented with only one download: "SU Patch [BYJ1]: WLS PATCH SET UPDATE 10.3.6.0.6 (Patch) p17071663_1036_Generic.zip"
This time around I get a "Error: /Stage[main]/Bsu/Orawls::Bsu[BYJ1]/Exec[exec bsu ux BYJ1]/returns: change from notrun to 0 failed: Command exceeded timeout" after the extraction of the patchset.
For the OSB version, refer to:
https://github.com/matthewbaldwin/vagrant-osb
which is really well documented.
PS Important: don't run "vagrant up" because the node1 will copydomain from admin node BEFORE this is ready. You should first do "vagrant up admin", then "vagrant up node1" and vagrant up node2". After the "vagrant up admin" the admin server is running and accessble from the host machine with http://10.10.10.10:7001/console/ (this is not a global ip, it will work only locally)
Tuesday, November 12, 2013
Puppet instant weblogic installation module in 30 seconds
define weblogic($user = 'soa') {
$optpath = '/opt'
$nmPort = '5556'
$oracle_home = "${optpath}/oracle"
$user_home = '/home/'
$bea_home = "${oracle_home}/fmw11_1_1_5"
$java_home = "/${optpath}/oracle/java"
$java_home_target = '/usr/lib/jvm/jre-1.6.0-sun.x86_64/bin/java'
$software_home = '/opt/oracle/software'
group { 'soa' :
ensure => 'present',
}
user { 'soa':
ensure => 'present',
gid => 'soa',
home => "/${user_home}/soa",
password => 'changeme',
managehome => true,
}
# create /home/soa?/.bashrc
file { "/${user_home}/${user}/.bashrc":
ensure => present,
content => template('weblogic/bashrc.erb'),
require => User["${user}"],
owner => "${user}",
}
file { "${optpath}":
ensure => 'directory',
owner => 'root',
group => 'root',
mode => '0755'
}
file { ["${software_home}"]:
ensure => 'directory',
owner => 'soa',
require => User['soa'],
}
file { "${java_home}":
ensure => link,
target => "${java_home_target}"
}
file { ["${oracle_home}", "${bea_home}"]:
ensure => 'directory',
owner => "${user}",
require => User["${user}"],
}
file { ["${optpath}/var/", "${optpath}/var/log/", "${optpath}/var/log/weblogic/", "${optpath}/var/log/weblogic/server",
"${optpath}/var/log/weblogic/nodemanager", "${optpath}/var/log/weblogic/scripts"]:
ensure => directory,
owner => $user,
require => User[$user],
}
# download WebLogic jars
file { 'wls1035_generic.jar':
ensure => present,
source => 'puppet:///modules/weblogic/wls1035_generic.jar',
target => "${software_home}/wls1035_generic.jar",
}
file { "${software_home}/silent_${user}.xml":
ensure => present,
content => template('weblogic/silent.xml.erb'),
require => File["${oracle_home}"]
}
# execute silent installation
exec { 'weblo-silent-install':
command => "java -d64 -jar wls1035_generic.jar -mode=silent -silent_xml=silent_${user}.xml",
cwd => "${software_home}",
path => '/usr/bin:/bin',
creates => "${bea_home}/registry.xml",
require => [Nexus::Download['wls1035_generic.jar'], File["${software_home}/silent_${user}.xml"], ],
}
}
all you need to do is to provide a silent.xml.erb and a bashrc.erb in the templates folder, plus the wls1035_generic.jar file in the files folder of the same weblogic module.
It might not be elegant but it's really simple and it works.
Subscribe to:
Posts (Atom)


