Sunday, May 19, 2019

JAXRS all-in-one

All annotations are summarized here https://dzone.com/articles/introduction-to-restful-web-service-a-jax-rs-speci

"Base URL for any web application is:

http://:/

Append tag from web.xml with Base URL that forms:

http://://

Append class-level @Path(“class-level”) annotation and then method-level @Path(“method-level”) annotation that forms:

http://:////"





https://dzone.com/articles/an-introduction-to-jax-rs-annotations-part-1

https://dzone.com/articles/what-are-jax-rs-annotations


https://readlearncode.com/java-ee/what-is-javax-ws-rs-core-context-httpservletresponse-and-httpservletrequest/


https://readlearncode.com/java-ee/what-are-the-jax-rs-annotations-get-post-path-applicationpath/


https://dzone.com/articles/what-is-javaxwsrscorecontext



@ApplicationPath("api")
extends Application

@Path("/books")

@GET
@Produces(MediaType.APPLICATION_JSON)

@POST
@Consumes(MediaType.APPLICATION_JSON)

@PUT
@HEAD

@DELETE

@Path("{isbn}")

@PathParam("isbn") String isbn

@QueryParam

@OPTIONS

@javax.ws.rs.HeaderParam
javax.ws.rs.core.Response

javax.ws.rs.core.SecurityContext



It's all very clear apart @Context... read here to understand @Context
https://readlearncode.com/java-ee/what-is-javax-ws-rs-core-context-httpheaders-and-uriinfo/

this for SecurityContext : https://readlearncode.com/java-ee/what-is-javax-ws-rs-core-context-securitycontext/


and some more coding examples here https://www.mkyong.com/webservices/jax-rs/get-http-header-in-jax-rs/




Friday, May 17, 2019

More interesting readings on quarkus and microprofiles

I reveive from my friend Rob:

This will soon overhaul Spring Boot. Look how they fast grow with integrating all enterprise patterns on Quarkus. This community is the driver https://smallrye.io/

Look at this suberb documentation

https://quarkus.io/guides/, almost covering everything you need

https://quarkus.io/guides/logging-guide
https://quarkus.io/guides/jwt-guide
https://quarkus.io/guides/transaction-guide

and then this amazing reactive streaming doc

https://quarkus.io/guides/kafka-guide


this will leave spring boot behind really soon


Worth reading is also https://www.baeldung.com/quarkus-io

Interesting also eclipse.microprofile https://www.baeldung.com/eclipse-microprofile running on Open Liberty



Thursday, May 16, 2019

Jooq and QueryDSL as alternatives to JPQL, Panache,

HQL (and JPQL) both suck because they are not statically typed..."lack of type safety and absence of static query checking" "concatenation of strings which is usually very unsafe "

"Criteria Query API ended up very verbose and practically unreadable. "


https://www.jooq.org/

"jOOQ generates Java code from your database and lets you build type safe SQL queries through its fluent API. "

So this is no ORM framework, it uses your DB as it is, it simply allows you to write safer SQL queries directly in a fluent Java API. No Hocus-Pocus, it's a 1-to-1 mapping between DB and Java.

Here some examples https://www.jooq.org/doc/3.11/manual-single-page/#jooq-in-7-steps



http://www.querydsl.com/

how to use it https://www.baeldung.com/querydsl-with-jpa-tutorial and https://www.baeldung.com/intro-to-querydsl



If you want to use Native SQL or JPQL in Spring: https://www.baeldung.com/spring-data-jpa-query


This is the horribly verbose JPA Criteria API https://www.baeldung.com/hibernate-criteria-queries
" the main and most hard-hitting advantage of Criteria queries over HQL is the nice, clean, Object Oriented API."




Apache SSL and ciphersuites

Useful commands and links

https://httpd.apache.org/docs/2.4/mod/mod_ssl.html#sslciphersuite


openssl ciphers -v

http://www.openssl.org/docs/apps/ciphers.html

httpd -v
httpd -V

https://httpd.apache.org/docs/2.4/ssl/ssl_howto.html

openssl s_client -connect 129.123.123.112:443

yum list all
yum install nmap


nmap -p 443 --unprivileged -script ssl-enum-ciphers 129.123.123.112

https://www.tecklyfe.com/nmap-script-test-ssl-versions-cipher-suites/

SSLCipherSuite HIGH:!aNULL:!MD5

SSLCipherSuite HIGH:!aNULL:!MD5:!SSLv3:!TLSv1

PORT    STATE SERVICE
443/tcp open  https
| ssl-enum-ciphers: 
|   SSLv3: No supported ciphers found
|   TLSv1.0: No supported ciphers found
|   TLSv1.1: No supported ciphers found
|   TLSv1.2: 
|     ciphers: 
|       TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 - strong
|       TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 - strong
|       TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 - strong
|       TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 - strong
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 - strong
|       TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 - strong
|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 - strong
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 - strong
|       TLS_RSA_WITH_AES_128_CBC_SHA256 - strong
|       TLS_RSA_WITH_AES_128_GCM_SHA256 - strong
|       TLS_RSA_WITH_AES_256_CBC_SHA256 - strong
|       TLS_RSA_WITH_AES_256_GCM_SHA384 - strong
|     compressors: 
|       NULL
|_  least strength: strong





https://en.wikipedia.org/wiki/Évariste_Galois

talking about GCM ciphers https://en.wikipedia.org/wiki/Galois/Counter_Mode






yarn react and materialui

open cmd with admin rights:
choco install yarn
close this cmd and open a "normal" cmd

yarn create react-app app
cd app
yarn add bootstrap@4.1.3 react-cookie@3.0.4 react-router-dom@4.3.1 reactstrap@6.5.0
yarn add @material-ui/core
yarn start


in App.js:


import React, {Component} from 'react';
import './App.css';
import Button from '@material-ui/core/Button';
import Table from '@material-ui/core/Table';
import TableRow from '@material-ui/core/TableRow';
import TableCell from '@material-ui/core/TableCell';




Friday, May 10, 2019

CDI in IntelliJ Java Enterprise projects

I was following this https://www.jetbrains.com/help/idea/creating-and-running-your-first-java-ee-application.html tutorial to deploy a Primefaces application to Wildfly 16, but I kept getting an error


unable to find cdi beanmanager

I have tried adding Maven support and adding the dependency

<dependencies>

<dependency>
<groupId>javax.enterprise</groupId>
<artifactId>cdi-api</artifactId>
<version>1.2</version>
</dependency>

</dependencies>


to no avail.

Funnily, setting webapp version="2.3" instead of version="4.0" in web.xml fixes the problem.


Then I added CDI support as shown here:

https://www.jetbrains.com/help/idea/context-and-dependency-injection-cdi.html

and using version="4.0" and things are working.

good old friend JMeter

JMeter is one of those primitive animals that never goes extinct.

Decent presentation:




Excellent comparison with SoapUI
https://octoperf.com/blog/2018/06/05/jmeter-vs-soapui/

JMeter is good for a VERY quick and dirty PERFORMANCE test of a service.


Incorporating JMeter Performance tests in Maven builds: https://www.ubik-ingenierie.com/blog/shift-left-performance-tests-jmeter-maven/ and this is the plugin https://github.com/jmeter-maven-plugin/jmeter-maven-plugin/wiki





Some books on the topic:




Sai Matam, Jagdeep Jain - Pro Apache JMeter_ Web Application Performance Testing Apress (2017)

Bayo Erinle - JMeter Cookbook Packt Publishing (2014)

Bayo Erinle - Performance Testing with JMeter 3 Enhance the performance of your web application Packt Publishing (2017)





GraalVM native image

download GraalVM CE and unzip it to /home/centos/graalvm-ce-19.0.0/

export JAVA_HOME=/home/centos/graalvm-ce-19.0.0/
export GRAALVM_HOME=/home/centos/graalvm-ce-19.0.0/

cd /home/centos/graalvm-ce-19.0.0/bin
./java -version
sudo ./gu install native-image

vi HelloWorld.java

public class HelloWorld {
        public static void main(String[] args) {
            System.out.println("hello world");
        }
}



./javac HelloWorld.java

time ./java HelloWorld


hello world

real 0m0.074s
user 0m0.057s
sys 0m0.019s



./native-image -H:+ReportExceptionStackTraces HelloWorld

Build on Server(pid: 6249, port: 45750)
[helloworld:6249]    classlist:     299.78 ms
[helloworld:6249]        (cap):     359.96 ms
[helloworld:6249]        setup:     479.50 ms
Error: Basic header file missing (). Make sure headers are available on your system.
com.oracle.svm.core.util.UserError$UserException: Basic header file missing (). Make sure headers are available on your system.
 at com.oracle.svm.core.util.UserError.abort(UserError.java:65)
 at com.oracle.svm.hosted.c.CAnnotationProcessor.reportCompilerError(CAnnotationProcessor.java:138)
 at com.oracle.svm.hosted.c.codegen.CCompilerInvoker.compileAndParseError(CCompilerInvoker.java:74)
 at com.oracle.svm.hosted.c.CAnnotationProcessor.compileQueryCode(CAnnotationProcessor.java:131)
 at com.oracle.svm.hosted.c.CAnnotationProcessor.process(CAnnotationProcessor.java:84)
 at com.oracle.svm.hosted.c.NativeLibraries.finish(NativeLibraries.java:311)
 at com.oracle.svm.hosted.NativeImageGenerator.processNativeLibraryImports(NativeImageGenerator.java:1523)
 at com.oracle.svm.hosted.NativeImageGenerator.setupNativeLibraries(NativeImageGenerator.java:997)
 at com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:829)
 at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:521)
 at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:441)
 at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
 at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
 at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
 at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
 at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Error: Image build request failed with exit status 1



sudo yum install glibc-devel
sudo yum install zlib-devel
sudo yum install gcc

./native-image HelloWorld

time ./helloworld
hello world

real 0m0.012s
user 0m0.001s
sys 0m0.011s




References:

https://www.graalvm.org/






Monday, May 6, 2019

Accelerate

https://www.kobo.com/us/en/ebook/accelerate-7




This is an excellent book - not only about technology but mostly about culture, leadership, teamwork, innovation.

I don't share the enthusiastic vision of the author for which life is about serving corporations to beat competition and cut jobs. However it's an educational book.


JDK maven Nexus and HTTPS

If your Nexus repository uses certificates signed by your own Root CA, chances are that a JDK doesn't trust them.

Then when you run

mvn package

you get


sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException:
unable to find valid certification path to requested target


Go to your JAVA_HOME\jre\lib\security folder, where the cacerts file is located, and issue

keytool -list -v -keystore cacerts

enter "changeit" as password

this shold tell you all your trusted CAs

You should import your own CA certificate to into this keystore.


I have tried also setting:


set MAVEN_OPTS="-Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true -Dmaven.wagon.http.ssl.ignore.validity.dates=true"

but it disn't work for me.




Panache as a wrapper for Hibernate

https://quarkus.io/guides/hibernate-orm-panache-guide

https://developers.redhat.com/courses/quarkus/effective-data-hibernate-and-panache-quarkus/

The product seems very well conceived, it really streamlines your JPA code.

One more aspect where the Java world has completely screwed up, is the 20 different ways you can implement DB queries....
ah if only ORM had been embedded into the language from the beginning, we would be dealing with a single persistence framework.



Thursday, May 2, 2019

Enabling Swagger in a Java EE application

This should be enough:

0) add Maven dependencies:

com.wordnik:swagger-jaxrs_2.10:1.3.1


1) with an Application, register the Swagger jaxrs resources:

import java.util.Set;

import javax.ws.rs.ApplicationPath;
import javax.ws.rs.core.Application;

@ApplicationPath("/rest")
public class JaxRsActivator extends Application {

 @Override
 public Set<Class<?>> getClasses() {
  Set<Class<?>> resources = new java.util.HashSet<>();
  resources.add(com.wordnik.swagger.jaxrs.listing.ApiListingResource.class);
  resources.add(com.wordnik.swagger.jaxrs.listing.ApiDeclarationProvider.class);
  resources.add(com.wordnik.swagger.jaxrs.listing.ApiListingResourceJSON.class);
  resources.add(com.wordnik.swagger.jaxrs.listing.ResourceListingProvider.class);
//  resources.add(BearerTokenFilter.class);
  addRestResourceClasses(resources);
  return resources;
 }

 private void addRestResourceClasses(Set<Class<?>> resources) {
  resources.add(BlaService.class);
  resources.add(MumbleService.class);
 }

}




2) register the Swagger configuration:


import javax.servlet.ServletConfig;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;

import com.wordnik.swagger.config.ConfigFactory;
import com.wordnik.swagger.config.ScannerFactory;
import com.wordnik.swagger.config.SwaggerConfig;
import com.wordnik.swagger.jaxrs.config.DefaultJaxrsScanner;
import com.wordnik.swagger.jaxrs.reader.DefaultJaxrsApiReader;
import com.wordnik.swagger.reader.ClassReaders;

@WebServlet(name = "SwaggerJaxrsConfig", loadOnStartup = 1)
public class SwaggerJaxrsConfig extends HttpServlet {

 @Override
 public void init(ServletConfig servletConfig) {
  try {
   super.init(servletConfig);
   SwaggerConfig swaggerConfig = new SwaggerConfig();
   ConfigFactory.setConfig(swaggerConfig);
   swaggerConfig.setBasePath("/rest");
   swaggerConfig.setApiVersion("2.1");
   ScannerFactory.setScanner(new DefaultJaxrsScanner());
   ClassReaders.setReader(new DefaultJaxrsApiReader());
  } catch (ServletException e) {
   System.out.println(e.getMessage());
  }
 }
}





3) then you can annotare your services methods:

import javax.inject.Inject;
import javax.ws.rs.Consumes;
import javax.ws.rs.GET;
import javax.ws.rs.HeaderParam;
import javax.ws.rs.NotAuthorizedException;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import com.wordnik.swagger.annotations.Api;
import com.wordnik.swagger.annotations.ApiOperation;
import com.wordnik.swagger.annotations.ApiParam;
import com.wordnik.swagger.annotations.ApiResponse;
import com.wordnik.swagger.annotations.ApiResponses;


@Path("/pippo")
@Consumes({ MediaType.APPLICATION_JSON })
@Produces({ MediaType.APPLICATION_JSON })
@Api(value = "/pippo", description = "REST service end-points exposed to Pippo")
public class PippoService {
 @GET
 @Path("/profiles")
 @Produces(MediaType.APPLICATION_JSON)
 @ApiOperation(value = "All Pippos", notes = "List all Pippos")
 @ApiResponses(value = { @ApiResponse(code = 200, message = "OK"), @ApiResponse(code = 500, message = "Server error, check log files") })
 public List listPippobyQuery(
                 @ApiParam(value = "the hostname where Pippo resides") @QueryParam("hostname") String hostName) {

  return listPippo(hostName);
 }
}











Monday, April 29, 2019

JAXRS client with Jersey

As usual we start from the excellent Baeldung https://www.baeldung.com/jersey-jax-rs-client

The working solution is in https://github.com/vernetto/jerseyrest and the client is in the test directory (Jersey is provided with "test" scope, while the "main" can be built with maven and deployed to WebLogic for instance)

The project is to turn the Service in a set of services with different types of payloads (JSON, XML, binary) and different marshaling/unmarshaling frameworks (Jexson, GSON...) , then run some performance test (using junit5) under different load conditions.





Swagger for Java EE reasteasy or jersey

Making Swagger work in Spring is very easy.

You can use it also in Java EE (Wildfly, WebLogic, even Tomcat!!! )

but all the documentation I have found is a bit outdated:

I have tried APIEE https://dzone.com/articles/apiee-an-easy-way-to-get-swagger-on-java-ee and https://github.com/phillip-kruger/apiee but it's not showing my services and I see no error.

Some other outdated articles:

https://github.com/swagger-api/swagger-core/wiki/Swagger-Core-Jersey-2.X-Project-Setup-1.5



https://github.com/swagger-api/swagger-core/wiki/Swagger-Core-RESTEasy-2.X-Project-Setup-1.5

I have no time now, but if I need to write REST services in Java EE I would not live without Swagger...

Gradle deploy to Nexus

mkdir /home/centos/gitclones/gradletest
cd /home/centos/gitclones/gradletest
gradle init --type java-application

gradle build

vi build.gradle
add to plugins:
id 'maven'

maven Plugin for Gradle is documented here https://docs.gradle.org/current/userguide/maven_plugin.html

gradle install

ls -ltra /home/centos/.m2/repository/gradletest/unspecified/
total 4956
-rw-r--r-- 1 centos docker 2371522 Apr 29 13:23 gradletest-unspecified.zip
-rw-r--r-- 1 centos docker 2693120 Apr 29 13:23 gradletest-unspecified.tar
-rw-r--r-- 1 centos docker     752 Apr 29 13:23 gradletest-unspecified.jar
-rw-r--r-- 1 centos docker     756 Apr 29 13:23 gradletest-unspecified.pom


cat /home/centos/.m2/repository/gradletest/unspecified/gradletest-unspecified.pom

  <groupId></groupId>
  <artifactId>gradletest</artifactId>
  <version>unspecified</version>


group and version are missing!

then you can follow the steps here https://medium.com/dot-debug/deploying-artifacts-to-maven-using-gradle-b669acc1b6f8

you define the group and version in your projects' build.gradle, you define nexus username/password/url in ~/.gradle/gradle.properties and you use the maven-publish plugin to publish to Nexus





IntelliJ IDEA Essentials




this is a really precious book, very focused and essential, to learn very practical tricks on how to use Intellij effectively




Sunday, April 28, 2019

JPG to PDF conversion in Java

https://gist.github.com/gholker/9a6b68ae51b3bef8931b946958dd81f2


Create a Spring Initializer App (plain vanilla)


add this dependency: com.itextpdf:itextpdf:5.5.13

run this code


package org.pierre.jpgtopdf;

import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

import com.itextpdf.text.Document;
import com.itextpdf.text.Image;
import com.itextpdf.text.PageSize;
import com.itextpdf.text.pdf.PdfWriter;

import java.io.File;
import java.io.FileOutputStream;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;


@SpringBootApplication
public class JpgtopdfApplication implements CommandLineRunner {

    public static void main(String[] args) {
        SpringApplication.run(JpgtopdfApplication.class, args);
    }

    @Override
    public void run(String... args) throws Exception {
        File root = new File("D:\\pierre\\tasse2019\\");
        String outputFile = "output.pdf";
        List<String> files = new ArrayList<String>();
        String[] alljpgs = root.list((dir, name) -> name.endsWith("jpg"));
        files.addAll(Arrays.asList(alljpgs));

        Document document = new Document();
        PdfWriter.getInstance(document, new FileOutputStream(new File(root, outputFile)));
        document.open();
        for (String f : files) {
            document.newPage();
            Image image = Image.getInstance(new File(root, f).getAbsolutePath());
            image.setAbsolutePosition(0, 0);
            image.setBorderWidth(0);
            image.scaleAbsolute(PageSize.A4);
            document.add(image);
        }
        document.close();
    }
}




It's amazingly fast!



CDI and javax.transaction.TransactionScoped

I was wondering how TransactionScoped works, and I stumbled upon this wonderful example:

https://www.byteslounge.com/tutorials/java-ee-cdi-transactionscoped-example

This also is a great explanation on Injected beans https://www.byteslounge.com/tutorials/java-ee-cdi-dependency-injection-inject-tutorial

It's so clearly explained that I was deeply impressed by this www.byteslounge.com
Gonçalo Marques. We need more of this stuff. Often people post partial, overcomplicated examples.... Gonçalo instead goes straight to the point.

Tuesday, April 23, 2019

Jenkins enable project based security

Sometimes you share a Jenkins instance amongst several projects (IMHO this is bad practice, each project should have its own Jenkins to minimize interference)

This is how to do it (copied from https://stackoverflow.com/questions/32111825/jenkins-how-to-set-authorization-on-project-basis )


a) make sure Matrix Authorization Strategy Plugin is installed (Manage Jenkins/Manage Plugins/Installed Plugins)

b) "Manage Jenkins", "Configure Global Security", add the target user to the "Project-based Matrix Authorization Strategy ",
add the target user with permissions "Overall/Read and Job/Read"

c) on the main page, select the target project, "Enable project-based security", add the target user and on the right, click on the "grant all permissions" button

at this point the user has login and can administer the target project, but only view other projects.

Jenkins would be a much better tool if all these configuration operations could be easily scriptable. Nowadays it's just a huge clickodrome and very awkward to manage, you have to wade through zillion of configuration pages and unless you are really experienced it's sometimes frustrating ... you don't even have a "search" functionality for configuration options, you have to remember all locations by heart...

Sunday, April 21, 2019

Simple Spring Boot and React working example

https://developer.okta.com/blog/2018/07/19/simple-crud-react-and-spring-boot

Matt Raible is a boss and he is so straight to the point and precise.

Remember that if you have in your pom.xml the dependency spring-boot-starter-security, then Spring Boot will enable security by default with username "user" and a dynamically generated password:

https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-security.html

so for the time being I have excluded this dependency.

This https://www.baeldung.com/spring-security-login should explain more how to handle web security with the great fluent api provided by Spring.

Friday, April 19, 2019

Intellij and WebLogic

https://www.jetbrains.com/help/idea/configuring-and-managing-application-server-integration.html



Wednesday, April 17, 2019

never copy/paste from outlook or Word

Microsoft sucks




The command on the left was copied from an email (Outlook). And it was failing, returning EVERYTHING, not only config.xml.

Then I have typed all over again (right pane) and it was working

Visually they look absolutely the same, even in Notepad++. But a closer inspection reveals hidden chars.

Copying and pasting from Outlook or Word has caused more victims than the 1919 Spanish Flu.

I remember a production delivery failed because someone copied a command from a Word document where Word had capitalized a property name.



CompletableFuture

https://www.callicoder.com/java-8-completablefuture-tutorial/

package com.example.demo;

import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

import java.util.concurrent.CompletableFuture;
import java.util.concurrent.TimeUnit;

@SpringBootApplication
public class DemoApplication implements CommandLineRunner {

    public static void main(String[] args) {
        SpringApplication.run(com.example.demo.DemoApplication.class, args);
    }

    @Override
    public void run(String... args) throws Exception {
        CompletableFuture<String> completableFuture = new CompletableFuture<String>();
        completableFuture.complete("Future's Result");

        String result = completableFuture.get();
        System.out.println(result);


        CompletableFuture<Void> future = CompletableFuture.runAsync(new Runnable() {
            @Override
            public void run() {
                // Simulate a long-running Job
                try {
                    TimeUnit.SECONDS.sleep(1);
                } catch (InterruptedException e) {
                    throw new IllegalStateException(e);
                }
                System.out.println("I'll run in a separate thread than the main thread.");
            }
        });
        // Block and wait for the future to complete
        future.get();
    }

}







Ruminations about Jenkins

The jobs are defined in here:

$JENKINS_HOME/jobs

For every project (item), there is a folder.

In every folder there is a config.xml file which hopefully should contain the entire project definition.

My assumption is that one should simply save the config.xml, and this is enough to recreate the project elsewhere.

I am worried because the Project definitions in Jenkins are not saved in bitbucket,
and we don’t have an automated way to export them from PROD and import them to a UAT instance.

One COULD tar the $JENKINS_HOME/jobs folder, but it’s very bulky.


[jobs]$ cd myproject/

[myproject]$ ls -ltra

drwxr-xr-x. 3 pippo pippogroup 4096 Jul 18 2016 modules
-rw-r--r--. 1 pippo pippogroup 6 Apr 25 2018 nextBuildNumber
lrwxrwxrwx. 1 pippo pippogroup 26 Apr 25 2018 lastSuccessful -> builds/lastSuccessfulBuild
lrwxrwxrwx. 1 pippo pippogroup 22 Apr 25 2018 lastStable -> builds/lastStableBuild
drwxr-xr-x. 5 pippo pippogroup 4096 Apr 25 2018 workspace
drwxr-xr-x. 24 pippo pippogroup 4096 Apr 25 2018 builds
-rw-r--r--. 1 pippo pippogroup 7257 Apr 25 2018 config.xml
drwxr-xr-x. 5 pippo pippogroup 4096 Apr 25 2018 .
drwxr-xr-x. 360 pippo pippogroup 20480 Mar 18 15:06 ..


especially the workspace is a PIG

So for the time being I will simply tar up all the config.xml and untar them in UAT:


cd $JENKINS_HOME/jobs

find . –maxdepth 2 –name config.xml | tar cvf /var/tmp/alljenkinsconfig.tar –T –

(maxdepth is important to avoid picking up files coming from the workspaces)

However we should really really IMHO push all those config.xml to bitbucket, regularly – ideally automatically whenever someone changes a config.xml:

git init
#set origin and remote
find . –maxdepth 2 –name config.xml –exec git add {} \;
git commit -m "blablabla"
git push


Incidentally, many project folders contain spaces, which makes it much trickier to write scripts to manipulate them.

I am not in favor of Capital Punishment, apart from cases when people create folders containing spaces.


Jenkins sucks anyway. Design is from Napoleonian Era, UI made by a freak, configuration freakishly XML based without fluent Administration Groovy API,
actually in the old times they were much better at designing stuff, think of the Tour Eiffel or the Pyramids and the Coliseum.
Nowadays any idiotic monkey can code freakish products like Maven or Jenkins and become a celebrity.



Thread dumps analysis

this post provides useful insights on how to detect issues from a Thread Dump:

http://allthingsmdw.blogspot.com/2012/02/analyzing-thread-dumps-in-middleware_08.html

For a quick thread dump consolidation one can use https://spotify.github.io/threaddump-analyzer/ or https://www.jetbrains.com/help/idea/analyzing-external-stacktraces.html (both are quite equivalent)....

Problem is that these tools are Application Server agnostic and don't tell you what is normal and what is not. I think some AI should be added to the tool, plus some graphical rendering for instance of the lock analysis.

When I have time I want to look into https://fastthread.io/

Anyway advanced thread analysis is a skill in itself and should be done with proper tooling... manually sorting stuff can be overwhelming for the regular human being.

Saturday, April 13, 2019

Mockito revisited

https://github.com/mockito/mockito


I have used Mockito in 2010. Then I never did SERIOUS Java development any more (shame on me...)

Yet I strongly believe that a solid implementation of Mocks is the foundation of healthy, fully testable software.
If you can't entirely automate your application tests, you are playing with your life.


https://search.maven.org/artifact/org.mockito/mockito-core/2.26.0/jar

<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <version>2.26.0</version>
</dependency>









https://github.com/in28minutes/MockitoTutorialForBeginners



https://www.baeldung.com/mockito-annotations

source code is here https://github.com/eugenp/tutorials/tree/master/testing-modules/mockito


https://static.javadoc.io/org.mockito/mockito-core/2.27.0/org/mockito/Mock.html

@Mock
List mockedList;


https://static.javadoc.io/org.mockito/mockito-core/2.27.0/org/mockito/Spy.html

@Spy
List spiedList = new ArrayList();


https://static.javadoc.io/org.mockito/mockito-core/2.27.0/org/mockito/ArgumentCaptor.html

@Captor
ArgumentCaptor argCaptor;


I am getting : Illegal reflective access by org.mockito.internal.util.reflection.AccessibilityChanger
Hopefully Mockito folks will fix this one da.







Wednesday, April 10, 2019

Gradle plugin for Jenkins

Maven Sucks, so we welcome Gradle - it can't possibly be worse than Maven!

https://wiki.jenkins.io/display/JENKINS/Gradle+Plugin

it seems that Gradle Plugin is installed as part of the "common plugins". I use an old Jenkins 2.138.


export GRADLE_HOME=/home/centos/gradle/gradle-4.8
#make sure $GRADLE_HOME/bin is in $PATH
cd /home/centos/gitclones
git clone https://github.com/jitpack/gradle-simple
cd gradle-simple/
gradle clean
gradle build
#check if jar was built
ls -ltra build/libs


I configure a freestyle Jenkins item to checkout from Github and "clean build" as gradle tasks...
Then I inspect /home/centos/.jenkins/workspace/gradletest/build/libs/ and the jar has been built there... great!


If I want to copy the plugin to another machine, it's in /home/centos/.jenkins/plugins/gradle.jpi

The MANIFEST.MF says:

Manifest-Version: 1.0
Plugin-Dependencies: structs:1.3
Long-Name: Gradle Plugin
Compatible-Since-Version: 1.0
Plugin-Developers: Stefan Wolf:wolfs:
Group-Id: org.jenkins-ci.plugins
Extension-Name: gradle
Plugin-Version: 1.31
Jenkins-Version: 1.651.3
Url: https://wiki.jenkins.io/display/JENKINS/Gradle+Plugin
Short-Name: gradle







Tuesday, April 9, 2019

Effective Java third edition


the source code:

https://github.com/jbloch/effective-java-3e-source-code






*clone (p. 86)
copy constructor/ copy factory
compareTo()
java.util.Comparator.comparingInt + thenComparingInt


immutable objects

composition over inheritance

default methods

generics and collections

unbounded wildcard types (<?>)

PECS producer extends consumer super

isAnnotationPresent
@ExceptionTest
@Repeatable


java.util.function
Operator
Predicate
Function
Supplier
Consumer



Monday, April 8, 2019

Generics and collections

this one has no compilation errors:

Set a;
Set<Object> b;
Set<? super Object> c;

a = new HashSet();
b = new HashSet<>();
c = new HashSet<>();

c = a;
c = b;
b = a;
b = c;
a = c;
a = b;



but if I use

Set<Object> b;
Set<?> c;

then "c = b" gives compilation error

Error:(26, 13) java: incompatible types: java.util.Set cannot be converted to java.util.Set


This is interesting, since ? means "any Object and subclasses"... while "? super Object" means "Object and its superclasses"
but Object has no superclasses, so effectively it's only Object.


If I use:
b = new HashSet<>();
Set<?? extends Object> c;

then
b = c;
fails with the same error:

Error:(26, 13) java: incompatible types: java.util.Set cannot be converted to java.util.Set


The raw type a can be assigned all the time to all the others.


similarly, collections are INVARIANT:
Set<Number> numset = new HashSet<>();
Set<Integer> intset = new HashSet<>();
Set<?> allset = new HashSet<>();

numset = intset; // INVALID
intset = numset; // INVALID
allset = numset; // VALID
numset = allset; // INVALID



but arrays are COVARIANT:

Number[] numarr = new Number[1];
Integer[] intarr = new Integer[1];
numarr = intarr; // VALID
intarr = numarr; // INVALID










Sunday, April 7, 2019

Amazing brushes by Reto




a colleague of mine, Reto, is a passionate photographer and Photoshop artist , here some links to his creations:

https://creativemarket.com/xresch/2642460-1500-Brushes-Megapack

https://creativemarket.com/xresch

https://affinity.serif.com/en-gb/store/product/180-smoke-and-cloud-brushes/




Intellij , maven and the source or target java language level

I create a new Java "maven" project, set it to Java 12 (ctrl-alt-shift-S, MOdules, Language level), but when I build I get:

"Warning:java: source value 1.5 is obsolete and will be removed in a future release"


Apparently this is due to the prehistoric, nonsensical Maven Compiler Plugin using Java 1.5 by default

https://stackoverflow.com/questions/27037657/stop-intellij-idea-to-switch-java-language-level-every-time-the-pom-is-reloaded


you can either insert in the pom.xml :
<properties>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
    </properties>



or
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.8.0</version>
            <configuration>
                <source>1.11</source>
                <target>1.11</target>
            </configuration>
        </plugin>

    </plugins>

</build>



Remember also to make sure that ctrl-alt-S (Settings) Settings -> Build, Execution, Deployment -> Build Tools -> Maven -> Importing is set to use JDK 12. So complicated. So brittle. So Maven.

Working with Maven is like traveling back to prehistoric times, like the Mural de la Prehistoria in Cuba, https://en.wikipedia.org/wiki/Vi%C3%B1ales_Valley



developers on the right about to be eaten by Maven on the left


Priceless String manipulation Intellij plugin

ctrl-alt-S, then "Plugins" search for "String Manipulations"; install and reboot.

for instance, if you need to "escape XML" (that is transform reserve chars in "% bla ;" sequences :

File/new scratch file (or ctrl-alt-shift-ins), paste your XML, select all, right click "String manipulation", "escape xml"


Normally I used Notepad++ and the "XML Tools Plugin", but this seems more useful.





Wednesday, April 3, 2019

git synchronize repositories

It happens that an external Git repository has to be mirrored internally in a corporate git repository. This cloning and mirroring should include branches, tags, commit history etc. and be one-way (internal changes are lot upon synchronization)

Luckily you don't have to write special scripts, all is provided by git:


https://help.github.com/en/articles/duplicating-a-repository



Let's make an experiment

in github I create a "gitclonesource" and a "gitclonedestination" empty repositories

mkdir gitclonetests
cd gitclonetests


#let's prepare the source
mkdir gitclonesource
cd gitclonesource
git init
echo "ciao" > README.md
git add README.md
git commit -m "first commit"
git remote add origin https://github.com/vernetto/gitclonesource.git
git push -u origin master

#create branch
git checkout -b mybranch
echo "hello" >> mybranch.txt
git add mybranch.txt
git commit -am "added mybranch.txt"
git push --set-upstream origin mybranch

#create tag
git checkout master
git tag -a v1.4 -m "my version 1.4"
git push --tags



#now clone source and push to destination with all branches and tags
cd ..
#this will create a folder gitclonesource.git
git clone --bare https://github.com/vernetto/gitclonesource.git
#when you cd, you will see a message BARE:master if using Git bash
cd gitclonesource.git/
git push --mirror https://github.com/vernetto/gitclonedestination

Counting objects: 7, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (7/7), 600 bytes | 200.00 KiB/s, done.
Total 7 (delta 0), reused 0 (delta 0)
To https://github.com/vernetto/gitclonedestination
 * [new branch]      master -> master
 * [new branch]      mybranch -> mybranch
 * [new tag]         v1.4 -> v1.4




and, lo and behold, in the gitclonedestination I can find my branch and the tag! All cloned in one go!

at this point you can cleanup the gitclonedestination.git:

cd ..
rm -rf gitclonedestination.git








Monday, April 1, 2019

WebLogic Security documentation

https://docs.oracle.com/middleware/12213/wls/INTRO/security.htm#INTRO232



https://docs.oracle.com/cd/E19798-01/821-1841/girbe/index.html JACC "Java Authorization Contract for Containers"

https://docs.oracle.com/javase/8/docs/api/java/security/Permission.html java.security.Permission

https://docs.oracle.com/javase/8/docs/api/java/lang/SecurityManager.html java.lang.SecurityManager





Java Authentication Service Provider Interface for Containers (JASPIC)




Training Material:

https://www.udemy.com/how-to-secure-oracle-weblogic-12c/


https://www.udemy.com/oracle-weblogic-12c-for-administrators/


Old book https://www.amazon.com/Securing-Weblogic-Server-Luca-Masini/dp/1849687781





Best Spring books

reblogging for my own reference, I am planning to read all those 5 books...

https://javarevisited.blogspot.com/2018/04/5-spring-framework-books-experienced-Java-developers-2018.html

- Spring in Action 5
- Cloud Native Java
- Learning Spring Boot 2.0
- Spring 5 Recipes
- Spring Microservices in Action



I am already going through https://www.udemy.com/spring-framework-5-beginner-to-guru/ but I find it incredibly non-concise - lot of repeated stuff, very lengthy and boring coding sessions where you don't really see much new, lot of unnecessary verbosity. Not my style, I like crisp, focused, simple examples and just fundamental facts and concepts.


Saturday, March 30, 2019

Java EE 8 Application Development book

David R. Heffelfinger - Java EE 8 Application Development-Packt Publishing (2017)



I am going through the examples of the book, using WLS 12.2.1.3 (it supports only Java EE 7, unfortunately), and IntelliJ 2019.1

here how to setup your environment (very easy!) :

https://www.jetbrains.com/help/idea/creating-and-running-your-first-java-ee-application.html#bfda7423


Code is here https://github.com/PacktPublishing/Java-EE-8-Application-Development


You can run also on Wildfly 16 https://wildfly.org/downloads/ which supports Java EE 8


Java EE 8 Javadoc https://javaee.github.io/javaee-spec/javadocs/index.html?overview-summary.html

Friday, March 29, 2019

bash script cheatsheet




Tuesday, March 26, 2019

Camel rediscovered

https://cleverbuilder.com/articles/camel-tutorial/

testing rest with  http://rest-assured.io/

https://github.com/rest-assured/rest-assured/wiki/Usage#json-schema-validation

http://camel.apache.org/enterprise-integration-patterns.html


http://camel.apache.org/components.html





Old posts:

http://www.javamonamour.org/2017/10/camel-in-action-second-edition.html


http://www.javamonamour.org/2017/08/apache-camel-training.html

http://www.javamonamour.org/2011/04/camel-is-cool.html

http://www.javamonamour.org/2010/07/enterprise-integration-patterns.html

Helm tutorials on Katacoda

I have been through some youtube tutorials on Helm but I found them too blablaistic. I like essential, crisp and down to earth presentations, not all-encompassing philosophical sermons.

https://www.katacoda.com/aptem/scenarios/helm


curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get | bash
helm init
helm repo update
kubectl get pods -n kube-system
helm version

in Chart.yaml there is basically nothing (version and chartname)

in equirements.yaml there are dependencies

in templates/configmap.yaml the ConfigMap

in templates/service.yaml the Service Definition (with port/nodePort)

in templates/pv.yaml the PersistenVolume


helm dependency update lets-chat

helm inspect lets-chat

helm install lets-chat --name demo -f my_values.yml --debug --dry-run

helm install lets-chat --name demo --namespace demo -f my_values.yml

helm list

helm status demo

kubectl get pods -n demo

sed -i 's/0.4.7/0.4.6/g' lets-chat/Chart.yaml

helm upgrade demo lets-chat --set replicas=2

helm history demo

helm rollback demo 1





maven the reference guide the complete reference

https://books.sonatype.com/mvnref-book/pdf/mvnref-pdf.pdf




https://www.baeldung.com/maven usual excellent Baeldung tutorial


https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html to understand those obfuscated concepts of "lifecycle", "phase", "goal"






Monday, March 25, 2019

Book: java by comparison by Simon Harrer, Jörg Lenhard, Linus Dietz

Interesting book, concise and a breeze to read

https://pragprog.com/book/javacomp/java-by-comparison



Source code is here https://pragprog.com/titles/javacomp/source_code

My first (not very functional) implementation of FizzBuzz:

import java.util.stream.IntStream;

public class ConsoleBasedFizzBuzz implements FizzBuzz {
    public static void main(String[] args) {
        FizzBuzz fizzBuzz = new ConsoleBasedFizzBuzz();
        fizzBuzz.print(1, 100);
    }

    private static void accept(int i) {
        StringBuffer result = new StringBuffer();
        if (i % 3 == 0) result.append("Fizz");
        if (i % 5 == 0) result.append("Buzz");
        if (result.length() == 0) result.append(i);
        System.out.println(result);
    }

    @Override
    public void print(int from, int to) {
        IntStream.range(from, to).forEach(ConsoleBasedFizzBuzz::accept);
    }
}



"Any fool can write code that a computer can understand. Good programmers write code that humans can understand."


Avoid Unnecessary Comparisons
Avoid Negations
Return Boolean Expressions Directly
Simplify Boolean Expressions
Avoid NullPointerException in Conditionals
Avoid Switch Fallthrough
Always Use Braces
Ensure Code Symmetry
Replace Magic Numbers with Constants
Favor Enums Over Integer Constants
Favor For-Each Over For Loops
Avoid Collection Modification During Iteration
Avoid Compute-Intense Operations During Iteration
Group with New Lines
Favor Format Over Concatenation
Favor Java API Over DIY
Remove Superfluous Comments
Remove Commented-Out Code
Replace Comments with Constants
Replace Comments with Utility Methods
Document Implementation Decisions
Document Using Examples
Structure JavaDoc of Packages
Structure JavaDoc of Classes and Interfaces
Structure JavaDoc of Methods
Structure JavaDoc of Constructors
Use Java Naming Conventions
Follow Getter/Setter Conventions for Frameworks
Avoid Single-Letter Names
Avoid Abbreviations
Avoid Meaningless Terms
Use Domain Terminology
Fail Fast
Always Catch Most Specific Exception
Explain Cause in Message
Avoid Breaking the Cause Chain
Expose Cause in Variable
Always Check Type Before Cast
Always Close Resources
Always Close Multiple Resources
Explain Empty Catch
Structure Tests Into Given-When-Then
Use Meaningful Assertions
Expected Before Actual Value
Use Reasonable Tolerance Values
Let JUnit Handle Exceptions
Describe Your Tests @DisplayName @Disabled
Favor Standalone Tests
Parametrize Your Tests @ParameterizedTest @ValueSource
Cover the Edge Cases
Split Method with Optional Parameters
Favor Abstract Over Concrete Types
Favor Immutable Over Mutable State
Combine State and Behavior
Avoid Leaking References (defensive copying)
Avoid Returning Null
Favor Lambdas Over Anonymous Classes
Favor Method References Over Lambdas
Avoid Side Effects
Use Collect for Terminating Complex Streams
Avoid Exceptions in Streams
Favor Optional Over Null (Optional.ofNullable())
Avoid Optional Fields or Parameters
Use Optionals as Streams

Google Java Style Guide https://google.github.io/styleguide/javaguide.html

Automate Your Build
Favor Logging Over Console Output
Minimize and Isolate Multithreaded Code
Use High-Level Concurrency Abstractions
Speed Up Your Program


import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.function.Executable;

import java.io.IOException;

public class JunitAssertionsTest {

    @Test
    public void testException() {
        Executable when = () -> pippo();
        Assertions.assertThrows(IOException.class, when);
    }

    private void pippo() throws IOException {
        System.out.println("running");
        throw new IOException("ciao");
    }
}



spotbugs former findbugs

https://spotbugs.readthedocs.io/en/latest/maven.html#goals-of-spotbugs-maven-plugin

<plugin>
 <groupId>com.github.spotbugs</groupId>
 <artifactId>spotbugs-maven-plugin</artifactId>
 <version>3.1.11</version>
 <dependencies>
  <!-- overwrite dependency on spotbugs if you want to specify the version 
   of spotbugs -->
  <dependency>
   <groupId>com.github.spotbugs</groupId>
   <artifactId>spotbugs</artifactId>
   <version>3.1.12</version>
  </dependency>
 </dependencies>
</plugin>


mvn spotbugs:spotbugs

this generates a file with a report:

target/spotbugsXml.xml

the report is hard to read honestly, but there are many interesting remarks





Sunday, March 24, 2019

Java 12 is out

Download : https://www.oracle.com/technetwork/java/javase/downloads/jdk12-downloads-5295953.html


Documentation :
https://docs.oracle.com/en/java/javase/12/


quick "what's new in Java 12" my the good Marco Behler




Nothing really revolutionary as far as I can see....mostly "enhanced switches" are cool


To use Java 12 you must install Intellij 2019.1 https://blog.jetbrains.com/idea/2019/01/intellij-idea-2019-1-early-access-program-is-open/

Saturday, March 23, 2019

Friday, March 22, 2019

Running Windows in Windows

You can download an .ova appliance for VirtualBox here https://developer.microsoft.com/en-us/windows/downloads/virtual-machines , unzip it, then open it in VistualBox , create a "shared" folder to exchange files with your host....

a very powerful and safe way to experiment without risking the health of your main Windows installation.

The images last only for 3 months, that's good enough for me :)


I see that also a Hyper-V image is available...


Thursday, March 21, 2019

Quarkus and Microservices

I receive this email from my friend Rob and I gladly publish it:


I see a lot of migration work going on in the future from existing JEE solutions, broken down to microservices and moving to native k8s images compiled with GraalVM.

The complete JEE programming model gets a fresh look towards microservices.

https://microprofile.io/

Go through this presentation

https://docs.google.com/presentation/d/1KsVjbmGcZuFCtx5F7ZeCwogM6VPCC6VwwLYFQelb8H8/edit#slide=id.g4f0ac3570c_11_0

get the ebook

https://microprofile.io/2019/02/25/building-an-api-backend-with-microprofile-ebook/

and the github

https://github.com/cicekhayri/ebook-Building-an-API-Backend-with-MicroProfile

and finally this will all go to this, quarkus "Quarkus is a Kubernetes Native Java framework tailored for GraalVM and HotSpot, crafted from best-of-breed Java libraries and standards."

https://microprofile.io/2019/03/07/next-generation-kubernetes-native-java-framework-implements-eclipse-microprofile/


This is the fastest realtime hot replace developer solution I have seen "quarkus:dev", this outperforms spring boot devtools

https://quarkus.io/guides/getting-started-guide



Tuesday, March 12, 2019

IntelliJ, Eclipse and Java 11


https://blog.jetbrains.com/idea/2018/06/java-11-in-intellij-idea-2018-2/

you must install IDEA 2018.2 or greater. 2018.1 will not work, it supports only Java 10. It's a 500 MB monster so be patient.

https://www.jetbrains.com/idea/download/ download 2018.3.5 here

then configure the platform JDK with your installed JDK 11 (of course you must have installed this JDK from Oracle or OpenJDK!), with ALT-SHIFT-CONTROL-S (project structure/SDKs)



then configure Java 11 level (File/Project Structure/Modules and File/Project Structure/Project)



also don't forget the bytecode level (File/Settings/Java Compiler):





If you get the "invalid source release: 11" , it's possible that you misconfigured the "platform JDK"




For Eclipse it's much easier: install Eclipse 2018-09 (4.9), then install this add-on:

https://marketplace.eclipse.org/content/java-11-support-eclipse-2018-09-49

make sure you configure the JDK in the "installed JRA" preferences tab,

and when you create a new project, select jdk11 and create the module-info by default.
All works fine.




GNOME desktop on Centos 7 docker

https://linuxconfig.org/how-to-install-gui-gnome-on-centos-7-linux-system

open Git Bash

winpty docker run -ti --name centos centos bash
#run all the commands to install GNOME
exit
docker start centos
winpty docker exec -ti centos bash

#uptime gives you the number of days the HOST was up, not the container
uptime





Monday, March 11, 2019

yipee.io, the online kubernetes yaml generator

I agree with every word written here:

https://yipee.io/wp-content/uploads/2018/10/yipee-whitepaper-oct-2018.pdf


beyond a very basic deployment, working directly with yaml files is suicidal.

I will explore this yipee and see if I can use it regularly, as replacement for a really primitive "vi mypod.yml"



Sunday, March 10, 2019

Kubernetes Java client to generate yaml files for you

If you - like me - hate having to type YAML by hand, you can take advantage of a pre-built K8S Model and YAML serialization tool:

https://github.com/fabric8io/kubernetes-client

<dependency>
    <groupId>io.fabric8</groupId>
    <artifactId>kubernetes-client</artifactId>
    <version>4.1.3</version>
</dependency>


import io.fabric8.kubernetes.api.model.Pod;
import io.fabric8.kubernetes.api.model.apps.DeploymentBuilder;
import io.fabric8.kubernetes.client.DefaultKubernetesClient;
import io.fabric8.kubernetes.client.KubernetesClient;
import io.fabric8.kubernetes.client.internal.SerializationUtils;

public class KubernetesClientTest {
 public static void main(String[] args) throws JsonProcessingException {
  String yaml = SerializationUtils.dumpAsYaml(new DeploymentBuilder().withNewSpec().endSpec().build());
  System.out.println(yaml);
  
  KubernetesClient client = new DefaultKubernetesClient();
  Pod pod = new Pod();
  System.out.println(pod);
  String podyaml = SerializationUtils.dumpAsYaml(pod);
  System.out.println(podyaml);
 }

}




The model is incredibly rich.... one has only to learn how to use it...

good presentation of Azure Kubernetes

https://youtu.be/gmN732qN1Gg


The session is available also here https://myignite.techcommunity.microsoft.com/sessions/65005
and here https://mediusproduction.blob.core.windows.net/presentations/Ignite2018/BRK2396.pptx you can download the slides (on the movie the resolution is quite lame)

What is https://en.wikipedia.org/wiki/Microsoft_Ignite ?


Azure DevSpaces https://docs.microsoft.com/en-us/azure/dev-spaces/
it's a direct tie between Visual Studio and AKS, with debugging of containers.


Azure Container Registry https://docs.microsoft.com/en-us/azure/container-registry/

Namespace-level RBAC security access, using AD groups.

Key Vaults contain Kubernetes and Application secrets (NO certificates in container!)

ACI https://docs.microsoft.com/en-us/azure/container-instances/ Azure Container Instances

OSB https://github.com/azure/open-service-broker-azure

Training modules:

https://docs.microsoft.com/en-us/learn/azure/ Introduction to Azure

https://docs.microsoft.com/en-us/learn/modules/welcome-to-azure/3-tour-of-azure-services an introduction to Azure Services

Friday, March 8, 2019

Quarkus Kubernetes and Katacoda

https://quarkus.io/guides/kubernetes-guide

I open a Katacoda lab with minikube in it https://www.katacoda.com/courses/kubernetes/launch-single-node-cluster

First step: https://quarkus.io/guides/getting-started-guide.html

git clone https://github.com/quarkusio/quarkus-quickstarts.git
cd getting-started
mvn compile quarkus:dev

now open a new terminal and "curl http://localhost:8080/hello"



cd quarkus-quickstarts/
cd getting-started-kubernetes/

#install graalvm
mkdir /root/graalvm
cd /root/graalvm
curl -L -o graalvm-ce-1.0.0-rc13-linux-amd64.tar.gz https://github.com/oracle/graal/releases/download/vm-1.0.0-rc13/graalvm-ce-1.0.0-rc13-linux-amd64.tar.gz
tar xvfz graalvm-ce-1.0.0-rc13-linux-amd64.tar.gz
export GRAALVM_HOME=/root/graal/graalvm-ce-1.0.0-rc13

mvn package -Pnative

here I get plenty of compilation errors, so I am giving up...


GraalVM is available here http://www.graalvm.org/downloads/
and more instructions here https://www.graalvm.org/docs/getting-started/





Kubernetes cheat sheet 3


Network Policies


kubectl get networkpolicy

kubectl describe networkpolicy

Name:         payroll-policy
Namespace:    default
Created on:   2019-03-08 08:47:51 +0000 UTC
Labels:       <none>
Annotations:  <none>
Spec:
  PodSelector:     name=payroll
  Allowing ingress traffic:
    To Port: 8080/TCP
    From:
      PodSelector: name=internal
  Allowing egress traffic:
    <none> (Selected pods are isolated for egress connectivity)
  Policy Types: Ingress


apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: internal-policy
  namespace: default
spec:
  podSelector:
    matchLabels:
      name: internal
  policyTypes:
  - Egress
  - Ingress
  ingress:
    - {}
  egress:
  - to:
    - podSelector:
        matchLabels:
          name: mysql
    ports:
    - protocol: TCP
      port: 3306

  - to:
    - podSelector:
        matchLabels:
          name: payroll
    ports:
    - protocol: TCP
      port: 8080




VOLUMES



https://portworx.com/basic-guide-kubernetes-storage/ good article

https://kubernetes.io/docs/concepts/storage/volumes/


https://kubernetes.io/docs/concepts/storage/persistent-volumes/

kind: PersistentVolume
apiVersion: v1
metadata:
  name: task-pv
spec:
  capacity:
    storage: 100Gi
  accessModes:
    - ReadWriteOnce
  awsElasticBlockStore:
    volumeID: vol-867g5kii
    fsType: ext4


https://github.com/kodekloudhub/kubernetes-challenge-1-wordpress







Wednesday, March 6, 2019

Exploring VS Code plugin for Kubernetes

Crafting yaml by hand is not ideal.

Install VS Code:

https://linuxize.com/post/how-to-install-visual-studio-code-on-centos-7/


the simply type "code" ( I could not find the shortcut in Applications/Programming)

ctrl-shift-X , type "Kubernetes", and install the first one (by Microsoft)

and also "Kubernetes Tools" https://marketplace.visualstudio.com/items?itemName=ms-kubernetes-tools.vscode-kubernetes-tools

https://github.com/Azure/vscode-kubernetes-tools source code here


https://code.visualstudio.com/docs/azure/kubernetes

Of course you must have AKS CLI installed https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-yum?view=azure-cli-latest

When you start "code" and click on the Kubernetes icon on the left, you get all the clusters listed in /home/centos/.kube/config file




Tuesday, March 5, 2019

Kubernetes cheat sheet 2

Namespaces


kubectl get pods --namespace=dev
kubectl get pods --namespace=default

kubectl config set-context $(kubectl config current-context) --namespace=dev


ConfigMap


kubectl create configmap myconfigmap --from-literal=APP_COLOR=blue
kubectl create -f myconfigmap.yml

apiVersion: v1
kind: ConfigMap
metadata:
  name: myconfigmap
data:
  APP_COLOR: blue
  APP_MODE: prod

then you inject into a container definition using
envFrom: 
- configMapRef
    name: myconfigmap

kubectl get configmaps
kubectl describe configmaps db-config


Secrets


kubectl create secret generic mysecret --from-literal=mykey=myvalue

apiVersion: v1
kind: Secret
metadata:
  name: app-secret
data:
  DBHost: mysql
  DBUser: root
  DBPassword: password


kubectl create -f secret_data.yaml


SECURITY

https://kubernetes.io/docs/tasks/configure-pod-container/security-context/

you can declare at Pod or container level:

spec:
  securityContext:
    runAsUser: 1000
    capabilities:
      add: ["MAC_ADMIN"]

#check which user runs the container
kubectl exec ubuntu-sleeper whoami



kubectl create serviceaccount dashboard-sa
kubectl get serviceaccount
kubectl describe serviceaccount dashboard-sa
kubectl describe secret dashboard-sa-account-token

curl https://myip/api -insecure --header "Authorization: Bearer PASTE_THE_TOKEN_HERE"

#change serviceaccount for a deployment
kubectl --record deployment.apps/web-dashboard set serviceaccount dashboard-sa



RESOURCES


resources:
  requests:
    memory: "1Gi"
    cpu: 1




Taints and Tolerations


kubectl taint nodes node-name key=value:taint-effect

taint-effect can be: NoSchedule, PreferNoSchedule, NoExecute

key=value can be app=blue


tolerations:
- key: "app"
operator: "Equal"
value: "blue"
effect: "NoSchedule"



to remove taint:

kubectl taint nodes master node-role.kubernetes.io/master:NoSchedule-


NODE SELECTOR


nodeSelector:
  size: Large


where size is a key and Large a value

to label a node:
kubectl label node mynode key=value

affinity:
 nodeAffinity:
   requiredDuringSchedulingIgnoredDuringExecution:
  nodeSelectorTerms:
  - matchExpressions:
    - key: color
   operator: In
   values:
   - blue
  


Readiness Probe


https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/

in the spec/containers/ section for each container:

readinessProbe:
  httpGet:
    path: /api/ready
    port: 8080
  initialDelaySeconds: 10
  periodSeconds: 5

beside httpGet you can have: "tcpSocket: port:", "exec: command:"



Liveness Probe


livenessProbe:
  httpGet:
    path: /api/ready
    port: 8080





Sunday, March 3, 2019

Kubernetes cheat sheet 1

https://kubernetes.io/docs/reference/kubectl/cheatsheet/

alias k=kubectl

PODS


https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/

kubectl run nginx --image=nginx


kubectl create -f nginx.yml


apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod
spec:
  containers:
    - name: nginx-container
      image: nginx


kubectl describe pods nginx-pod


kubectl get pods -o wide

kubectl edit pod nginx-pod

kubectl delete pod nginx-pod


kubectl get pod pod-name -o yaml > pod-definition.yaml

Replication Controller


kubectl create -f rc-definition.yml

apiVersion: v1
kind: ReplicationController
metadata:
  name: nginx
  spec:
    replicas: 3
    selector:
      app: nginx
    template:
      metadata:
        name: nginx
        labels:
          app: nginx
      spec:
        containers:
          - name: nginx
            image: nginx
            ports:
              - containerPort: 80

kubectl get rc


Replica Set


kubectl create -f rs-definition.yml

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: frontend
  labels:
    app: guestbook
    tier: frontend
spec:
# modify replicas according to your case
  replicas: 3
  selector:
    matchLabels:
      tier: frontend
  template:
    metadata:
      labels:
      tier: frontend
    spec:
      containers:
        - name: php-redis
          image: gcr.io/google_samples/gb-frontend:v3

kubectl get replicaset

kubectl describe replicaset

kubectl replace

kubectl scale --replicas=3 rs/myrs

kubectl get rs myrs -o yaml


Deployments


https://kubernetes.io/docs/concepts/workloads/controllers/deployment/

kubectl create -f mydeployment.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
  spec:
    replicas: 3
    selector:
      matchLabels:
        app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:1.7.9
          ports:
            - containerPort: 80

kubectl get all




An overall good explanation of Kubernetes is here https://dzone.com/storage/assets/11459286-dzone-refcard292-advancedkubernetes314.pdf




Excellent Kubernetes Developer Certification training on Udemy

https://www.udemy.com/certified-kubernetes-application-developer for only 11 USD !

Mumshad Mannambeth is a fantastic trainer, focusing on concepts but analyzing every detail in a very visual manner.



Hands On Full Stack Development with Spring Boot 2.0 and React by Juha Hinkula




you are warned: "incula" is a terrible word in Italian, so if you talk about this book with Italians you should be very careful.

https://github.com/PacktPublishing/Hands-On-Full-Stack-Development-with-Spring-Boot-2.0-and-React


Better to install MariaDB.

If you want to change root password:

cd D:\Program Files\MariaDB 10.3\bin

mysqladmin.exe --user=root --password=oldpassword password "newpassword"

to connect

mysql -u root -p

create database cardb



For chapter04, password can be replaced with "pippo" (bcrypted the value is $2a$04$ryGKh2NmcAj.dRzU/MDDVeTApJAMrfbyzVVGlclK8u/HLrTamBH5m )


curl -u admin:pippo http://localhost:8080/api (this will fail because you need a valid JWT token)

then the POST to localhost:8080/login must have a body {"username": "admin", "password": "pippo" }

and in the Authorization header you get Bearer eyJhbGciOiJIUzUxMiJ9.eyJzdWIiOiJhZG1pbiIsImV4cCI6MTU1MTY0Nzg3OX0.DlHNCsxwasv4wjZcar2afWWx5ispQWFfIfL6NEnsrDYucLqPQWO53WcmUUDJfw12t0ux-9P-HzwF8qKnd1SNDg




Thursday, February 28, 2019

CKA Certification (Kubernetes Administrator)

https://github.com/cncf/curriculum/blob/master/CKA_Curriculum_V1.12.0.pdf here the topics to be covered


I would start by reading the official doc https://kubernetes.io/docs/concepts/

Kubernetes Master : kube-apiserver, kube-controller-manager and kube-scheduler

Non-master node : kubelet, kube-proxy

Control Plane , kubectl,

etcd, kube-scheduler, kube-controller-manager

PodSpecs , Cluster DNS ,

(to be continued)



Monday, February 25, 2019

Viktor Farcic and Jenkins-X

https://leanpub.com/the-devops-2-6-toolkit

The ebook is available to read for free - the first 60 pages.

For a good overall presentation of JenkinsX https://opensource.com/article/18/11/getting-started-jenkins-x



az CLI for Azure Kubernetes Services (AKS)

https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-windows?view=azure-cli-latest

I am installing AZ on Windows. At the end of installation, I reboot (maybe not needed...)

az login -> this opens a browser window which automatically logs you in your AKS account

in return , you get a JSON with your account details. Nice.

[
{
"cloudName": "AzureCloud",
"id": "9ab0d5c-036-495-b6d-25b05be758",
"isDefault": true,
"name": "Free Trial",
"state": "Enabled",
"tenantId": "9dc2b0-897-4b6-adf1-8c5488ea413",
"user": {
"name": "pippo.pluto@hotmail.com",
"type": "user"
}
}
]

which can be obtained also by doing

az account show


I am stingy, so I first check my billing:

az billing invoice list

NotOptedIn - You are not allowed to download invoices. Please contact your account administrator (pippo.pluto@hotmail.com) to turn on access in the management portal for allowing to download invoices through the API.




So I log into https://portal.azure.com/


Be very careful not to use the Azure Cloud Shell available in the Browser UI... using it will create a Storage Account to host a shared drive needed by the Shell, and this will reduce to 1 month the (otherwise 1 year) trial account. So, just install the AZ cli on your machine and be happy.



To display a Resource Group (RG):
az group show --name myResourceGroup

To create a RG
az group create --name myResourceGroup --location "East US"






Saturday, February 2, 2019

JenkinsX and GKE on Centos

Install JX:

https://gist.github.com/vfarcic/8cef206b4df0b1bbec3060d1d45c2a80

mkdir -p ~/.jx/bin

curl -L https://github.com/jenkins-x/jx/releases/download/v1.3.634/jx-linux-amd64.tar.gz \
    | tar xzv -C ~/.jx/bin

export PATH=$PATH:~/.jx/bin

echo 'export PATH=$PATH:~/.jx/bin' \
>> ~/.bashrc



install gcloud:


https://cloud.google.com/sdk/docs/downloads-yum

tee -a /etc/yum.repos.d/google-cloud-sdk.repo << EOM
[google-cloud-sdk]
name=Google Cloud SDK
baseurl=https://packages.cloud.google.com/yum/repos/cloud-sdk-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
       https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOM


yum install google-cloud-sdk


create GKE cluster
jx create cluster gke
(this will install kubectl and try to install helm.... helm installation fails with
"error: error installing helm: failed to remove helm secrets: failed to run '/root/.jx/bin/helm plugin remove secrets' command in directory '', output: 'Error: Plugin: secrets not found': exit status 1"

So I download help here https://storage.googleapis.com/kubernetes-helm/helm-v2.12.3-linux-amd64.tar.gz
gunzip helm-v2.12.3-linux-amd64.tar.gz tar xvf helm-v2.12.3-linux-amd64.tar mv linux-amd64/helm /usr/local/bin/helm mv linux-amd64/tiller /usr/local/bin/tiller
I request an authorization code at https://accounts.google.com/o/oauth2/auth?
then run
gcloud auth login --brief
and paste the authorization code
jx install --provider=gke unable to load in-cluster configuration, KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT must be defined error: failed to create the kube client: unable to load in-cluster configuration, KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT must be defined
I do:
export KUBERNETES_SERVICE_HOST=cloud.google.com export KUBERNETES_SERVICE_PORT=443
then again "jx install --provider=gke" and I get
open /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory error: failed to create the kube client: open /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory
and of course also this fails
jx create cluster gke -n jx-rocks -p pippo -z us-east1-b -m n1-standard-2 --min-num-nodes 3 --max-num-nodes 5 --default-admin-password admin --default-environment-prefix jx-rocks
with this error
open /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory
Go to the following link in your browser:

    https://accounts.google.com/o/oauth2/auth?redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&prompt=select_account&response_type=code&client_id=32555940559.apps.googleusercontent.com&scope
=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fappengine.admin+https%3A%2F%2Fwww
.googleapis.com%2Fauth%2Fcompute+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Faccounts.reauth&access_type=offline


ERROR: There was a problem with web authentication.
ERROR: (gcloud.auth.login) EOF when reading a line
Enter verification code: Error: Command failed  gcloud auth login --brief
error creating cluster exit status 1
error: exit status 1

I have googled everywhere and I find no help... it seems that JX is still a niche product and immature..
I am giving up...
Maybe I will try some parameters https://jenkins-x.io/commands/jx_create_cluster_gke/
kubectl get ns The connection to the server localhost:8080 was refused - did you specify the right host or port?
what????






Wednesday, January 23, 2019

Minikube on Centos

I want to have a "mini" version of Kubernetes on my Centos laptop.

I would prefer not to install an Hypervisor, since k8s can use "--vm-driver=none"...
but I give it a try to install VirtualBox https://www.itzgeek.com/how-tos/linux/centos-how-tos/install-virtualbox-4-3-on-centos-7-rhel-7.html

After installing VirtualBox, I can proceed with minikube

https://kubernetes.io/docs/tasks/tools/install-minikube/


curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 \
&& chmod +x minikube

sudo cp minikube /usr/local/bin && rm minikube

minikube version
minikube version: v0.33.1

minikube start

I get "Error parsing version semver: Version string empty"


kubectl cluster-info
kubectl get nodes



I can try also with "minikube start --vm-driver=none", which however requires docker



Saturday, January 19, 2019

Coding and cooking


priceless!



Friday, January 18, 2019

Running Centos on Windows

once you have upgraded to Windows PRO, you can install https://docs.docker.com/docker-for-windows/install/ Docker for Windows

Then open your Git Bash shell and run:

winpty docker run --name centos -i -t -d centos

If the container is stopped, you can simply

docker start centos

and login again:

winpty docker exec -ti centos bash


So you can have a "REAL" Centos on Windows... cool...





VirtualBox and Hyper-V on Windows 10 PRO

I have recently installed Windows PRO. During installation of Hyper-V, it duly warns you that you will no longer be able to use VirtualBox. Which is true, because when I start a VM in VirtualBOx I get the message:


WHvCapabilityCodeHypervisorPresent is FALSE! Make sure you have enabled the 'Windows Hypervisor Platform' feature. (VERR_NEM_NOT_AVAILABLE).
VT-x is not available (VERR_VMX_NO_VMX).


Apparently all you need to do to re-enable VirtualBox is:


open CMD as Administrator
bcdedit /set hypervisorlaunchtype off

reboot your machine

To restore Hyper-V you simply run:
bcdedit /set hypervisorlaunchtype on
or
bcdedit /set hypervisorlaunchtype auto


I haven't tested yet but it should work IMHO



bcdedit is this https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/bcdedit-command-line-options

Windows Boot Manager
--------------------
identifier              {bootmgr}
device                  partition=\Device\HarddiskVolume2
path                    \EFI\Microsoft\Boot\bootmgfw.efi
description             Windows Boot Manager
locale                  fr-FR
inherit                 {globalsettings}
badmemoryaccess         Yes
default                 {current}
resumeobject            {b32f6aed-59a4-11e8-b3f7-b9390bbf0244}
displayorder            {current}
toolsdisplayorder       {memdiag}
timeout                 30

Windows Boot Loader
-------------------
identifier              {current}
device                  partition=C:
path                    \WINDOWS\system32\winload.efi
description             Windows 10
locale                  fr-FR
inherit                 {bootloadersettings}
recoverysequence        {b32f6aef-59a4-11e8-b3f7-b9390bbf0244}
displaymessageoverride  Recovery
recoveryenabled         Yes
badmemoryaccess         Yes
isolatedcontext         Yes
allowedinmemorysettings 0x15000075
osdevice                partition=C:
systemroot              \WINDOWS
resumeobject            {b32f6aed-59a4-11e8-b3f7-b9390bbf0244}
nx                      OptIn
bootmenupolicy          Standard
hypervisorlaunchtype    Auto