Saturday, September 23, 2017

Installing Docker on CentOS 7

from the excellent guide https://docs.docker.com/engine/installation/linux/docker-ce/centos/

sudo yum remove docker docker-common docker-selinux docker-engine
sudo yum install -y yum-utils   device-mapper-persistent-data   lvm2
sudo yum-config-manager --add-repo  https://download.docker.com/linux/centos/docker-ce.repo
sudo yum-config-manager --enable docker-ce-edge
sudo yum-config-manager --enable docker-ce-test
sudo yum install docker-ce
yum list docker-ce.x86_64  --showduplicates | sort -r
sudo systemctl start docker
sudo docker run hello-world
sudo docker run -it ubuntu bash
sudo yum makecache fast
sudo groupadd docker
echo $USER
sudo usermod -aG docker $USER
#(perform logout/login here)
groups
#"centos docker"
sudo systemctl enable docker
#"Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service."
sudo chkconfig docker on
env | grep DOCKER_HOST
#DOCKER_HOST should not be defined if you want to connect to local daemon
#to restart daemon:
sudo service docker restart




useful commands (from the presentation below):



https://www.youtube.com/watch?v=VeiUjkiqo9E&feature=youtu.be
docker ps -a
docker version
docker info
docker images

docker run ubuntu echo "hello world"

docker run -i -t ubuntu
exit

docker diff 82af4da88bd7
docker rm 042cbb043587

docker run -d ubuntu /bin/sh -c "while true; do echo hello world; date; sleep 2; done"
docker logs 7939ad46d57c
docker attach 7939ad46d57c
docker stop 7939ad46d57c

docker inspect 7939ad46d57c

docker commit -m "installed apache" 7939ad46d57c mynamespace/myimage
docker images

#login into https://hub.docker.com
docker login
docker push mynamespace/myimage

docker rmi 77bde6a39eda

#execute the Dockerfile in myfolder and build a new image
docker build -t mynamespace/myimage github.com/myrepo/myfolder

#run assigning a non default port
docker run -d -p :8000 mynamespace/myimage








here the link to shipyard https://github.com/shipyard/shipyard







Cool Tools (from JBoss Hacks)



be aware that the Kindle edition is royally scr...wed up, missing ALL the sample script in the text... what a pity... maybe the PDF version available here http://www.itbuzzpress.com/ebooks/jboss-org-hacks.html is better, no idea.

https://github.com/mgm3746/garbagecat to analyze gc logs

http://fastthread.io/index.jsp thread dump analysis

https://github.com/rparree/jboss-bash-completion jboss/wildfly bash completion

http://hawt.io/ web console rich of managing plugins

http://byteman.jboss.org/index.html byteman , bytecode manipulation tool

https://hub.docker.com/r/jboss/wildfly/ dockerhub wildfly image

https://developers.redhat.com/jboss-docker/?referrer=jbd docker jboss images

https://docs.jboss.org/wildfly/plugins/maven/latest/ wildfly maven plugin


Jboss Forge
examples available here https://github.com/fmarchioni/jbosshacks

Wildfly Swarm Project Generator http://wildfly-swarm.io/generator/


https://developers.redhat.com/products/rhamt/download/ windup / migration tool to migrate from other AS to JBoss



Friday, September 22, 2017

JBoss CLI sucks

Frankly the CLI hurts the eyes... someone should make it a bit more groovish... oh in fact here there is a guide how to do it http://www.mastertheboss.com/jboss-server/jboss-script/managing-wildfly-with-groovy-shell but it's a bit raw...

some recipes here:

https://docs.jboss.org/author/display/AS71/CLI+Recipes

at the end of the book "WildFly Configuration, Deployment, and Administration - Second Edition" there is also a useful CLI cheat sheet.



All Day DevOps on October 24, 2017 starting at 8:00am GMT

https://www.alldaydevops.com



register here https://www.alldaydevops.com/register

you can even watch the recorded session later (2016 recordings are available on their side...)

Sunday, September 10, 2017

Nexus Repository Migration

MIGRATING FROM 2.12.1 TO 2.14.5

Migrating from Nexus Repository Manager OSS 2.12.1-01 to the latest 3.0 version:

http://www.sonatype.org/nexus/2016/05/11/migrating-to-nexus-repository-3-easy-peasy/

https://www.youtube.com/watch?v=ba4VtpusRxo

beware: upgrade agent is part of 2.14 only! You should first upgrade to 2.14 then migrate to 3.0, as shown here https://www.youtube.com/watch?v=q6qkj8UiSRY

Very detailed instructions here https://help.sonatype.com/display/NXRM3/Upgrading

Download old releases here https://support.sonatype.com/hc/en-us/articles/218238798-Nexus-Repository-Manager-OSS-Download-Archives

I have installed Nexus 2.12 here: /home/centos/nexus2/nexus-2.12.0-01/bin

cd /home/centos/nexus2/nexus-2.12.0-01/bin
./nexus start


The URL is http://localhost:8081/nexus/ (in 3.X it's simply http://localhost:8081 )



In maven's settings.xml I had this
  <mirror>
        <!--This sends everything else to /public -->
        <id>nexus</id>
        <mirrorOf>*</mirrorOf>
        <url>http://localhost:8081/repository/maven-public/</url>
     </mirror>



this was valid for 3.5, in 2.12 the URL should be changed to http://localhost:8081/nexus/content/repositories/central/

I run a sample "mvn package" from "https://github.com/gabrielf/maven-samples" and I can observe that the repository "Maven Central" is filled with stuff: http://localhost:8081/nexus/service/local/feeds/recentlyCachedReleaseArtifacts, then I select the "Central" repo, "Browse Index" and I can see stuff in it (junit, hamcrest, codehaus...)

I want now to upgrade from 2.12.1 to 2.14.5 https://support.sonatype.com/hc/en-us/articles/213464198, which is the supported version to late upgrade to 3.5

I extract the 2.14.5 zip file in a temporary location and I copy the nexus-2.14.5-02 folder (NOT the sonatype-work folder!!!) to /home/centos/nexus2, so that the nexus-2.12 and nexus-2.14 share the same sonatype-work folder.

[centos@localhost nexus2]$ pwd
/home/centos/nexus2

[centos@localhost nexus2]$ ls -ltra
total 4
drwxr-xr-x.  8 centos centos  113 Dec 16  2015 nexus-2.12.0-01
drwxr-xr-x.  3 centos centos   37 Dec 16  2015 sonatype-work
drwxr-xr-x.  8 centos centos  113 Jul 25 12:39 nexus-2.14.5-02
drwx------. 33 centos centos 4096 Sep 10 19:28 ..
drwxrwxr-x.  5 centos centos   73 Sep 10 19:34 .



apparently the sonatype-work format is binary compatible among the 2.12 and 2.14.5 versions.

I stop nexus, make a backup copy of the conf folder:

cd /home/centos/nexus2
cp -R nexus-2.12.0-01/conf/ nexus-2.12.0-01/confBACKUP


apparently no further manual steps are required https://support.sonatype.com/hc/en-us/articles/213464338-Sonatype-Nexus-Upgrade-and-Compatibility-Notes

and now I start the new version of Nexus

cd nexus-2.14.5-02/bin
./nexus start
At a quick look, the content of the repository and an extra user I had created are preserved in the migration...

MIGRATING TO 3.5

Now I start nexus 3.5 side by side, on the same host, making sure I use a different number:

cd /home/centos/nexus3
grep -R 8081 *
nexus-3.5.1-02/etc/nexus-default.properties:application-port=8081


and I change that port to 18081

cd /home/centos/nexus3/nexus-3.5.1-02/bin
./nexus start


localhost:18081

things are working. I delete all pre-existing repositories

I follow all the steps as in https://help.sonatype.com/display/NXRM3/Upgrading and things work perfectly - using the "download" method (slowest)

I have tested the 3 methods and they all work, of course the file copy (hard link or not) is much faster than HTTP. If you wonder what a hard link is, read here https://askubuntu.com/questions/108771/what-is-the-difference-between-a-hard-link-and-a-symbolic-link



Saturday, September 9, 2017

Nexus and Maven (ST): setup

I have installed Maven (ST) in /home/centos/apache-maven-3.5.0/.

My m2 repo is in /home/centos/.m2/repository

To test the m2 repo, I "git clone https://github.com/gabrielf/maven-samples", then "mvn package"

"ls /home/centos/.m2/repository" shows that stuff is actually pulled from the maven central repo https://repo.maven.apache.org/maven2/ , you should see plenty of this in the logs :

"Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.5/maven-resources-plugin-2.5.jar"

I edit the settings in /home/centos/apache-maven-3.5.0/conf/settings.xml to incorporate this setup https://books.sonatype.com/nexus-book/3.0/reference/maven.html#maven-sect-single-group

I have installed Nexus in /home/centos/nexus30/. Make sure you increase the file descriptors before you start.

"cd /home/centos/nexus30/nexus-3.5.1-02/bin/" and "./nexus start".

Login at http://localhost:8081 using admin/admin123 , go to http://localhost:8081/#admin/repository/repositories and check that "maven central" is already preconfigured with http://localhost:8081/repository/maven-central/ URL. You can browse the Nexus content at http://localhost:8081/#browse/browse/components:maven-central (it should be empty at this stage)

Clear the local Maven (ST) repo "rm -rf /home/centos/.m2/repository/*" and run again "mvn package". This time Maven should retrieve the artifacts from Nexus:

Downloaded: http://localhost:8081/repository/maven-public/org/apache/maven/plugins/maven-resources-plugin/2.5/maven-resources-plugin-2.5.pom

Browsing again http://localhost:8081/#browse/browse/components:maven-central should show that all artifacts are cached in Nexus.

In case of issues, check the logs /home/centos/nexus30/sonatype-work/nexus3/log (nexus.log, request.log, jvm.log) for details.



PS: (ST) stands for STINKS . Maven STINKS like a wet rotten dirty skunk. Use Gradle instead.

Friday, September 8, 2017

Software Vulnerability Control with Sonatype products

Interesting introductory vide on the topic of Security in Software Supply Chain https://vimeo.com/217179090

Software Factory | Sonatype from Sonatype Nexus on Vimeo.



A really detailed presentation of the "Nexus Lifecycle" and "Nexus IQ server"





Software Supply Chain
Continuous Integration
Continuous Delivery
Release Automation Tool
DevOps Native Software Development
Nexus Firewall
Public repositories: Maven Central,

Sonar Security Rules: https://docs.sonarqube.org/display/SONAR/Security-related+rules

Software Weakness

Software Vulnerability

http://cwe.mitre.org/ Common Weakness Enumeration (common software security weaknesses) - very educational FAQ on Software Weakness here http://cwe.mitre.org/about/faq.html#A.1

https://www.first.org/cvss/ CVSS Common Vulnerability Scoring System, scores explained: https://www.first.org/cvss/specification-document

https://www.sonatype.com/nexus-lifecycle-tour-software-supply-chain-automation-sonatype Nexus lifecycle

https://nvd.nist.gov/ NVD is National Vulnerability Database - try searching for Bouncy Castle

Using "Application Health Check" to scan vulnerabilities:



I have read the devsecops Gartner report https://www.sonatype.com/devsecops also available here https://cdn2.hubspot.net/hubfs/1958393/White_Papers/devsecops_how_to_seamlessly__315283.pdf?t=1482418124868

Very interesting also the https://www.sonatype.com/ssc2017?hsCtaTracking=d915532d-28ac-4818-a2db-d4e8feb07036%7C61ff62a0-da6f-47d1-9591-c89200a00454 2017 State of the Open Software

See also the famous https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/owasptop10/OWASP%20Top%2010%20-%202013.pdf OWASP top 10 web application security risks. Number A9 says: "Components, such as libraries, frameworks, and other software modules, almost always run with full privileges. If a vulnerable component is exploited, such an attack can facilitate serious data loss or server takeover. Applications using components with known vulnerabilities may undermine application defenses and enable a range of possible attacks and impacts."

Repository Health Check RHC demo video here https://sonatype.wistia.com/medias/77jh7h47av

Result of a WebGoat Health Check https://clm.sonatype.com/reports/Sonatype/71fe66f5c3b540f09caa9ebf1f103e7a-8fae0/

Comparison of Free and Opensource Software Licenses https://en.wikipedia.org/wiki/Comparison_of_free_and_open-source_software_licenses

more videos on:

Brian Fox, Integration of Nexus Health Check with Eclipse https://www.youtube.com/watch?v=_wqxy5A7-oA&feature=youtu.be

Brian Fox, Nexus IQ Server email alerts on Weak Security https://www.youtube.com/watch?v=UHulp3PWqFk

Brian Fox, Nexus IQ Server, Define security policies https://www.youtube.com/watch?v=EnKHimjNjvQ



Wednesday, September 6, 2017

Dependency trees in Nexus and Maven: who uses what ?

Nexus plugin to display who uses a given artifact https://github.com/saleemshafi/nexus-artifact-usage-plugin (apparently the same info is built-in in Apache Archiva https://stackoverflow.com/questions/19392236/who-is-using-my-maven-artifact ). As suggested in SO, you should build a ├╝ber-POM containing all your projects, then generate the dependency tree.

Useful commands:

mvn dependency:tree -DoutputType=graphml -DoutputFile=dependency.graphml

(use also --debug in case of errors)

Here the documentation on Dependency Management in Maven https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html

Transitive Dependencies



Friday, September 1, 2017

Spring Framework Essentials course by Ken Kousen in Safari Books Online

Here is the code https://github.com/kousen/Spring-Framework-Essentials

Ken is a very talented teacher, who focuses on concepts rather than sheer data. Highly recommended.

The course is available only if you register in Safari Books Online https://www.safaribooksonline.com/, but there is a short term FREE evaluation account (no credit car required).