JDK maven Nexus and HTTPS

If your Nexus repository uses certificates signed by your own Root CA, chances are that a JDK doesn't trust them.

Then when you run

mvn package

you get


sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException:
unable to find valid certification path to requested target


Go to your JAVA_HOME\jre\lib\security folder, where the cacerts file is located, and issue

keytool -list -v -keystore cacerts

enter "changeit" as password

this shold tell you all your trusted CAs

You should import your own CA certificate to into this keystore.


I have tried also setting:


set MAVEN_OPTS="-Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true -Dmaven.wagon.http.ssl.ignore.validity.dates=true"

but it disn't work for me.

Gradle deploy to Nexus

mkdir /home/centos/gitclones/gradletest
cd /home/centos/gitclones/gradletest
gradle init --type java-application

gradle build

vi build.gradle
add to plugins:
id 'maven'

maven Plugin for Gradle is documented here https://docs.gradle.org/current/userguide/maven_plugin.html

gradle install

ls -ltra /home/centos/.m2/repository/gradletest/unspecified/

total 4956
-rw-r--r-- 1 centos docker 2371522 Apr 29 13:23 gradletest-unspecified.zip
-rw-r--r-- 1 centos docker 2693120 Apr 29 13:23 gradletest-unspecified.tar
-rw-r--r-- 1 centos docker     752 Apr 29 13:23 gradletest-unspecified.jar
-rw-r--r-- 1 centos docker     756 Apr 29 13:23 gradletest-unspecified.pom

cat /home/centos/.m2/repository/gradletest/unspecified/gradletest-unspecified.pom

  <groupId></groupId>
  <artifactId>gradletest</artifactId>
  <version>unspecified</version>

group and version are missing!

then you can follow the steps here https://medium.com/dot-debug/deploying-artifacts-to-maven-using-gradle-b669acc1b6f8

you define the group and version in your projects' build.gradle, you define nexus username/password/url in ~/.gradle/gradle.properties and you use the maven-publish plugin to publish to Nexus

Cleaning up old releases in Nexus 2 and 3

In Nexus 2 there is the task "remove releases from repository", where you can further refine the scope by specifying a "repository target" (=a filter).... problem is, the only option you have is the number of the most recent versions to retain.... there is no way to specify "delete all versions older than 2 years, but in any case keep the last 10".

I have written this Spring Boot utility https://github.com/vernetto/nexusclean to generate curl commands to delete artifacts based on multiple selection criteria. But it doesn't handle parent pom, which is a limitation.

Also, be aware that the curl command to delete a component, like

curl -u admin:admin123 -X "DELETE" -w "%{http_code}" http://localhost:8281/nexus/service/local/repositories/central/content/org/codehaus/plexus/plexus-classworlds/1.2-alpha-7

every time rebuilds the index, which is very CPU intensive. There is no way to batch multiple deletes in a single transaction.


In Nexus 3

https://github.com/xninjaxelitex/nexus3-cleanup-release-artifact

Nexus repo validates Docker images on production.cloudflare.docker.com

In Nexus logs I find a lot of calls to production.cloudflare.docker.com:



2018-11-05 14:13:43,416+0100 DEBUG [qtp72695066-56] ADV org.sonatype.nexus.httpclient.outbound - https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/4f/4fe2ade4980c2dda4fc95858ebb981489baec8c1e4bd282ab1c3560be8ff9bde/data?verify=1541426623-n%2BHbbRXN3Rr4k6Bxofrsv6tRVFw%3D > GET /registry-v2/docker/registry/v2/blobs/sha256/4f/4fe2ade4980c2dda4fc95858ebb981489baec8c1e4bd282ab1c3560be8ff9bde/data?verify=1541426623-n%2BHbbRXN3Rr4k6Bxofrsv6tRVFw%3D HTTP/1.1
2018-11-05 14:13:43,416+0100 DEBUG [qtp72695066-56] ADV org.sonatype.nexus.internal.httpclient.SharedHttpClientConnectionManager - Connection request: [route: {tls}->http://ourproxy:8080->https://production.cloudflare.docker.com:443][total kept alive: 1; route allocated: 0 of 20; total allocated: 3 of 200]
2018-11-05 14:13:43,417+0100 DEBUG [qtp72695066-56] ADV org.sonatype.nexus.internal.httpclient.SharedHttpClientConnectionManager - Connection leased: [id: 18][route: {tls}->http://ourproxy:8080->https://production.cloudflare.docker.com:443][total kept alive: 1; route allocated: 1 of 20; total allocated: 4 of 200]
2018-11-05 14:13:43,475+0100 DEBUG [qtp72695066-4467] ADV org.sonatype.nexus.httpclient.outbound - https://auth.docker.io/token?service=registry.docker.io&scope=repository:library/alpine:pull < HTTP/1.1 200 OK @ 129.8 ms 2018-11-05 14:13:43,475+0100 DEBUG [qtp72695066-4467] ADV org.sonatype.nexus.repository.docker.internal.DockerProxyFacetImpl - Response: HttpResponseProxy{HTTP/1.1 200 OK [Content-Type: application/json, Date: Mon, 05 Nov 2018 13:13:43 GMT, Transfer-Encoding: chunked, Strict-Transport-Security: max-age=31536000, Connection: Keep-Alive] ResponseEntityProxy{[Content-Type: application/json,Chunked: true]}}

we access internet via a Proxy Server ourproxy, which doesn't whitelist production.cloudflare.docker.com

each of them creates a file in $NEXUS_DATA/tmp/docker-content-validation-failures with an "access denied" message from ourproxy

Here https://forums.docker.com/t/corporate-firewall-remote-error-tls-handshake-failure/52965 they say we should also whitelist production.cloudflare.docker.com

I have no idea if the "docker pull" will fail, or if this "validation" can be disabled...




See also https://support.sonatype.com/hc/en-us/articles/115015442847-Whitelisting-Docker-Hub-Hosts-for-Firewalls-and-HTTP-Proxy-Servers

Nexus repository location of license

If you move your Nexus repository to a new machine, it's quite annoying to have to reinstall the license file.

The trick is to clone the folder ~/.java/.userPrefs, specifically the license is embedded in

~/.java/.userPrefs/com/sonatype/nexus/professional/prefs.xml

and it's common for Nexus2 and Nexus3

I had searched everywhere for a .lic file, to no avail of course.


This is also documented here https://support.sonatype.com/hc/en-us/articles/115000779668-Methods-to-Install-a-Nexus-Repository-Manager-3-License

Nexus Repository Security explained

https://help.sonatype.com/learning/repository-manager-3/security-essentials

RBAC (roles, privileges)

https://help.sonatype.com/learning/repository-manager-3/security-model---users%2C-roles%2C-and-privileges

Hammering Nexus 3.14

first, create an entry in your settings.xml for mvn to be able to authenticate in nexus

<server>
<id>nexus</id>
<username>admin</username>
<password>admin123</password>
</server>

then write this testnexus.sh script:

NEXUSHOME=/home/centos/nexus314
NEXUSLOG=$NEXUSHOME/sonatype-work/nexus3/log/nexus.log
rm $NEXUSLOG

cd $NEXUSHOME/nexus-3.14.0-04/bin

for i in `seq 1 1000`;
do
 echo "Iteration number " $i
 #make sure nexus is not running before you start it
 ./nexus stop
 ./nexus start
 while [ ! -f $NEXUSLOG ] ;
 do
  sleep 2
 done
 
 echo "logfile found"
 tail -f $NEXUSLOG | while read LOGLINE
 do
  [[ "$LOGLINE" == *"Started Sonatype Nexus OSS 3.14.0-04"* ]] && pkill -P $$ tail
 done
 #now that it's up, let's wait for some time
 
 echo "nexus started"
 NEXUSPID=`ps -e -o pid,command | grep org.sonatype.nexus.karaf.NexusMain | grep -v grep | awk '{print $1}'`
 echo NEXUSPID = $NEXUSPID
 #wait for a random time before we kill
 sleep $[ ( $RANDOM % 100 )  + 10 ]
 ps -ef | grep nexus
 echo killing $NEXUSPID
 kill -9 $NEXUSPID
 mv $NEXUSLOG  ${NEXUSLOG}.${i}
done

and here the 2 scripts to upload continuously stuff to nexus

testnexusupload1.sh

NEXUSHOME=/home/centos/nexus314

curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.pom
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.jar
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.md5
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.sha1
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.pom.sha1
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.pom.md5
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.jar.sha1
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.jar.md5


mvn deploy:deploy-file -DgroupId=commons-beanutils -DartifactId=commons-beanutils -Dversion=1.9.2 -DgeneratePom=true -Dpackaging=jar -DrepositoryId=nexus -Durl=http://localhost:8081/repository/maven-releases -Dfile=$NEXUSHOME/nexus-3.14.0-04/system/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.jar

testnexusupload2.sh

NEXUSHOME=/home/centos/nexus314

curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-codec/commons-codec/1.10/commons-codec-1.10.pom
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-codec/commons-codec/1.10/commons-codec-1.10.jar
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-codec/commons-codec/1.10/commons-codec-1.10.md5
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-codec/commons-codec/1.10/commons-codec-1.10.sha1
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-codec/commons-codec/1.10/commons-codec-1.10.pom.sha1
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-codec/commons-codec/1.10/commons-codec-1.10.pom.md5
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-codec/commons-codec/1.10/commons-codec-1.10.jar.sha1
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-codec/commons-codec/1.10/commons-codec-1.10.jar.md5


mvn deploy:deploy-file -DgroupId=commons-codec -DartifactId=commons-codec -Dversion=1.10 -DgeneratePom=true -Dpackaging=jar -DrepositoryId=nexus -Durl=http://localhost:8081/repository/maven-releases -Dfile=$NEXUSHOME/nexus-3.14.0-04/system/commons-codec/commons-codec/1.10/commons-codec-1.10.jar

testnexusupload3.sh (this will create always new jars)

NEXUSHOME=/home/centos/nexus314
for i in `seq 1 1000000`;
do
 mvn deploy:deploy-file -DgroupId=commons-beanutils -DartifactId=commons-beanutils -Dversion=1.10.${i} -DgeneratePom=true -Dpackaging=jar -DrepositoryId=nexus -Durl=http://localhost:8081/repository/maven-releases -Dfile=$NEXUSHOME/nexus-3.14.0-04/system/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.jar

 sleep 2
done

nohup ./testnexusupload1.sh > ./testnexusupload1.log 2>&1 &
nohup ./testnexusupload2.sh > ./testnexusupload2.log 2>&1 &
nohup ./testnexusupload3.sh > ./testnexusupload3.log 2>&1 &

or

setsid ./testnexusupload1.sh > ./testnexusupload1.log 2>&1 &
setsid ./testnexusupload2.sh > ./testnexusupload2.log 2>&1 &
setsid ./testnexusupload3.sh > ./testnexusupload3.log 2>&1 &


This way with a continuous cycle of restarts I can test the stability of the repository.
If the DB or shards are corrupted we will find out...



In fact, after some iteration I keep getting this error at startup:




2018-10-28 19:04:49,571+0100 INFO [elasticsearch[C1896386-DD1B2DA2-9E77236C-5051BC20-E1D39AE4][clusterService#updateTask][T#1]] *SYSTEM org.elasticsearch.cluster.metadata - [C1896386-DD1B2DA2-9E77236C-5051BC20-E1D39AE4] [2e9a1e67e8a325bcd6ee9f6790ff6c769e791d56] creating index, cause [api], templates [], shards [1]/[0], mappings [component]
2018-10-28 19:04:49,955+0100 INFO [elasticsearch[C1896386-DD1B2DA2-9E77236C-5051BC20-E1D39AE4][clusterService#updateTask][T#1]] *SYSTEM org.elasticsearch.cluster.routing.allocation - [C1896386-DD1B2DA2-9E77236C-5051BC20-E1D39AE4] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[2e9a1e67e8a325bcd6ee9f6790ff6c769e791d56][0]] ...]).
2018-10-28 19:04:50,065+0100 INFO [quartz-2-thread-1] *SYSTEM org.sonatype.nexus.repository.search.SearchFacetImpl - Rebuilding index of repository maven-central
2018-10-28 19:04:50,501+0100 INFO [elasticsearch[C1896386-DD1B2DA2-9E77236C-5051BC20-E1D39AE4][clusterService#updateTask][T#1]] *SYSTEM org.elasticsearch.cluster.metadata - [C1896386-DD1B2DA2-9E77236C-5051BC20-E1D39AE4] [73ae44bc066b6a7a33b4435641d8229b9b66495a] creating index, cause [api], templates [], shards [1]/[0], mappings [component]
2018-10-28 19:04:50,943+0100 ERROR [Thread-44 <command>sql.select from component where bucket = :bucket</command>] *SYSTEM com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage - Exception `32DA9B7B` in storage `plocal:/home/centos/nexus314/sonatype-work/nexus3/db/component`: 2.2.36 (build d3beb772c02098ceaea89779a7afd4b7305d3788, branch 2.2.x)
com.orientechnologies.orient.core.exception.OCommandExecutionException: Error on execution of command: sql.select from component where bucket = :bucket
DB name="component"
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.executeCommand(OAbstractPaginatedStorage.java:3421)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.command(OAbstractPaginatedStorage.java:3318)
at com.orientechnologies.orient.core.sql.query.OSQLQuery.run(OSQLQuery.java:78)
at com.orientechnologies.orient.core.sql.query.OSQLAsynchQuery.run(OSQLAsynchQuery.java:74)
at com.orientechnologies.orient.core.query.OQueryAbstract.execute(OQueryAbstract.java:33)
at com.orientechnologies.orient.core.sql.query.OSQLNonBlockingQuery$1.run(OSQLNonBlockingQuery.java:283)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException: null
at com.orientechnologies.orient.core.serialization.serializer.binary.impl.index.OCompositeKeySerializer.deserializeFromByteBufferObject(OCompositeKeySerializer.java:347)
at com.orientechnologies.orient.core.serialization.serializer.binary.impl.index.OCompositeKeySerializer.deserializeFromByteBufferObject(OCompositeKeySerializer.java:46)
at com.orientechnologies.orient.core.storage.impl.local.paginated.base.ODurablePage.deserializeFromDirectMemory(ODurablePage.java:166)
at com.orientechnologies.orient.core.index.sbtree.local.OSBTreeBucket.getKey(OSBTreeBucket.java:266)
at com.orientechnologies.orient.core.index.sbtree.local.OSBTreeBucket.find(OSBTreeBucket.java:134)
at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.findBucket(OSBTree.java:1679)
at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.access$2800(OSBTree.java:69)
at com.orientechnologies.orient.core.index.sbtree.local.OSBTree$OSBTreeCursorForward.next(OSBTree.java:1991)
at com.orientechnologies.orient.core.index.engine.OSBTreeIndexEngine$OSBTreeIndexCursor.nextEntry(OSBTreeIndexEngine.java:271)
at com.orientechnologies.orient.core.index.OIndexAbstractCursor.hasNext(OIndexAbstractCursor.java:83)
at com.orientechnologies.orient.core.index.OIndexChangesWrapper.hasNext(OIndexChangesWrapper.java:138)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.serialIterator(OCommandExecutorSQLSelect.java:1636)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.fetchFromTarget(OCommandExecutorSQLSelect.java:1585)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.fetchValuesFromIndexCursor(OCommandExecutorSQLSelect.java:2466)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchForIndexes(OCommandExecutorSQLSelect.java:2280)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchInClasses(OCommandExecutorSQLSelect.java:1017)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLResultsetAbstract.assignTarget(OCommandExecutorSQLResultsetAbstract.java:203)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.assignTarget(OCommandExecutorSQLSelect.java:527)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.executeSearch(OCommandExecutorSQLSelect.java:509)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.execute(OCommandExecutorSQLSelect.java:485)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:70)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.executeCommand(OAbstractPaginatedStorage.java:3400)
... 6 common frames omitted






This seems a known issue with a workaround/solution


https://issues.sonatype.org/browse/NEXUS-18036

java -jar ./nexus-3.14.0-04/lib/support/nexus-orient-console.jar

connect plocal:/home/centos/nexus314/sonatype-work/nexus3/db/component/ admin admin

Disconnecting from the database [null]...OK
Connecting to database [plocal:/home/centos/nexus314/sonatype-work/nexus3/db/component/] with user 'admin'...
2018-10-28 23:42:11:147 WARNI {db=component} Storage 'component' was not closed properly. Will try to recover from write ahead log... [OLocalPaginatedStorage]
2018-10-28 23:42:11:192 WARNI {db=component} Record com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OCheckpointEndRecord{lsn=LSN{segment=105, position=52}} will be skipped during data restore [OLocalPaginatedStorage]
2018-10-28 23:42:11:247 WARNI {db=component} Record OFuzzyCheckpointStartRecord{lsn=LSN{segment=105, position=59}} com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OFuzzyCheckpointStartRecord{lsn=null, previousCheckpoint=LSN{segment=105, position=28}} will be skipped during data restore [OLocalPaginatedStorage]
2018-10-28 23:42:11:248 WARNI {db=component} Record com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OFuzzyCheckpointEndRecord{lsn=LSN{segment=105, position=99}} will be skipped during data restore [OLocalPaginatedStorage]OK



repair database --fix-links

Repairing database...
- Removing broken links...
-- Done! Fixed links: 2, modified documents: 2
Repair database complete (0 errors)


rebuild index *


Rebuilding index(es)...
Rebuilt index(es). Found 4354 link(s) in 36.727001 sec(s).


Index(es) rebuilt successfully


disconnect
exit

Getting configuration information from Nexus FOR FREE

One VERY SIMPLE way to log all the Nexus 3 configuration in the nexus.log is to enable debug logging (Administration/Logging/ set the ROOT logger at DEBUG level), then navigate to the Repositories/Roles/Users administration page. You will see that precious JSON is logged in nexus.log, containing all the most minute details of the Repositories/Roles/User configuration.

Programmatically you can achieve the same with a HTTP POST to http://mynexusserver:8081/service/extdirect with this JSON request, Content-Type application/json

[{"action":"coreui_Blobstore","method":"read","data":null,"type":"rpc","tid":104},{"action":"coreui_Repository","method":"read","data":null,"type":"rpc","tid":105},{"action":"coreui_Repository","method":"readRecipes","data":null,"type":"rpc","tid":106},{"action":"coreui_Repository","method":"readReferences","data":[{"page":1,"start":0,"limit":25}],"type":"rpc","tid":107},{"action":"coreui_Repository","method":"read","data":null,"type":"rpc","tid":108}]

[{"action":"coreui_User","method":"read","data":[{"filter":[{"property":"source","value":"default"},{"property":"userId","value":""}],"page":1,"start":0,"limit":100}],"type":"rpc","tid":63},{"action":"coreui_User","method":"readSources","data":null,"type":"rpc","tid":64},{"action":"coreui_Role","method":"read","data":null,"type":"rpc","tid":65},{"action":"coreui_User","method":"read","data":[{"page":1,"start":0,"limit":100}],"type":"rpc","tid":66}]

It would be really cool if such API was made available through SwaggerUI... Nexus 3 is very very primitive as far as API is concerned.

Internally, those actions coreui_Blobstore , coreui_User, coreui_Repository are mapped this way:

action class 'org.sonatype.nexus.coreui.UserComponent' mapped to 'coreui_User'

action class 'org.sonatype.nexus.coreui.RepositoryComponent' mapped to 'coreui_Repository'

action class 'org.sonatype.nexus.coreui.BlobStoreComponent' mapped to 'coreui_Blobstore'


which are in the plugin/nexus-coreui-plugin area as Groovy classes (this for Nexus 3.10... I think in Nexus 3.13 things have changed..)

bintray endless redirection for repo.scala-sbt.org

The Scala folks running this repository don't seem to understand the importance of running their stuff behind a Proxy Server, so that all elements come from the same domain.... instead, they send you HTTP 302 (MOVED) and force you to hop from repo.scala-sbt.org to dl.bintray.com and then to akamai.bintray.com .... which is outrageously cumbersome if you run your Nexus repository from behind a Firewall, then you have to open individually each subdomain... otherwise you end up with the dreaded " HTTP/1.1 407 Proxy Authentication Required "



2018-10-16 16:21:19,993+0200 DEBUG [qtp72695066-17874] *UNKNOWN org.sonatype.nexus.httpclient.outbound - https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-native-packager/scala_2.12/sbt_1.0/1.3.4/jars/sbt-native-packager.jar > GET /scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-native-packager/scala_2.12/sbt_1.0/1.3.4/jars/sbt-native-packager.jar HTTP/1.1

2018-10-16 16:21:20,102+0200 DEBUG [qtp72695066-17874] *UNKNOWN org.sonatype.nexus.httpclient.outbound - https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-native-packager/scala_2.12/sbt_1.0/1.3.4/jars/sbt-native-packager.jar < HTTP/1.1 302 Moved Temporarily @ 108.4 ms 2018-10-16 16:21:20,102+0200 DEBUG [qtp72695066-17874] *UNKNOWN org.sonatype.nexus.httpclient.outbound - https://dl.bintray.com/sbt/sbt-plugin-releases/com.typesafe.sbt/sbt-native-packager/scala_2.12/sbt_1.0/1.3.4/jars/sbt-native-packager.jar > GET /sbt/sbt-plugin-releases/com.typesafe.sbt/sbt-native-packager/scala_2.12/sbt_1.0/1.3.4/jars/sbt-native-packager.jar HTTP/1.1

2018-10-16 16:21:20,193+0200 DEBUG [qtp72695066-17874] *UNKNOWN org.sonatype.nexus.httpclient.outbound - https://dl.bintray.com/sbt/sbt-plugin-releases/com.typesafe.sbt/sbt-native-packager/scala_2.12/sbt_1.0/1.3.4/jars/sbt-native-packager.jar < HTTP/1.1 302 @ 90.77 ms 2018-10-16 16:21:20,195+0200 DEBUG [qtp72695066-17874] *UNKNOWN org.sonatype.nexus.httpclient.outbound - https://akamai.bintray.com/3f/3f786b85a05e14c7ed5c24fbefbee19fbaa345e1db5d16fa9fa348c2a4e4789e?__gda__=exp=1539700400~hmac=9d5b250bffebb7cedf93ef28933b588c8788e9a7b20415dbe12e255435bf5ede&response-content-disposition=attachment%3Bfilename%3D%22sbt-native-packager.jar%22&response-content-type=application%2Fjava-archive&requestInfo=U2FsdGVkX1_tU3jkSNxcBb5JrmKzZS9pVZUOhb41Zj_AapUhiUGbx2fl-2B2bTDZvRaC1DmJfpw37KETEYGfMyBkdRqY-gknijMaFBx-HSJPsgC4aYd6Ejwpe2uzTSrTmhIF7HIgBbnG6hm8PxLHTvaGWPkWDnEacMK9-YMvxz8zphO4eHB46kniHIOc-vop&response-X-Checksum-Sha1=f8ab68145e3a23da45be0847f114065aa2fc0f54&response-X-Checksum-Sha2=3f786b85a05e14c7ed5c24fbefbee19fbaa345e1db5d16fa9fa348c2a4e4789e > GET /3f/3f786b85a05e14c7ed5c24fbefbee19fbaa345e1db5d16fa9fa348c2a4e4789e?__gda__=exp=1539700400~hmac=9d5b250bffebb7cedf93ef28933b588c8788e9a7b20415dbe12e255435bf5ede&response-content-disposition=attachment%3Bfilename%3D%22sbt-native-packager.jar%22&response-content-type=application%2Fjava-archive&requestInfo=U2FsdGVkX1_tU3jkSNxcBb5JrmKzZS9pVZUOhb41Zj_AapUhiUGbx2fl-2B2bTDZvRaC1DmJfpw37KETEYGfMyBkdRqY-gknijMaFBx-HSJPsgC4aYd6Ejwpe2uzTSrTmhIF7HIgBbnG6hm8PxLHTvaGWPkWDnEacMK9-YMvxz8zphO4eHB46kniHIOc-vop&response-X-Checksum-Sha1=f8ab68145e3a23da45be0847f114065aa2fc0f54&response-X-Checksum-Sha2=3f786b85a05e14c7ed5c24fbefbee19fbaa345e1db5d16fa9fa348c2a4e4789e HTTP/1.1

2018-10-16 16:21:20,204+0200 DEBUG [qtp72695066-17874] *UNKNOWN org.sonatype.nexus.httpclient.outbound - https://akamai.bintray.com/3f/3f786b85a05e14c7ed5c24fbefbee19fbaa345e1db5d16fa9fa348c2a4e4789e?__gda__=exp=1539700400~hmac=9d5b250bffebb7cedf93ef28933b588c8788e9a7b20415dbe12e255435bf5ede&response-content-disposition=attachment%3Bfilename%3D%22sbt-native-packager.jar%22&response-content-type=application%2Fjava-archive&requestInfo=U2FsdGVkX1_tU3jkSNxcBb5JrmKzZS9pVZUOhb41Zj_AapUhiUGbx2fl-2B2bTDZvRaC1DmJfpw37KETEYGfMyBkdRqY-gknijMaFBx-HSJPsgC4aYd6Ejwpe2uzTSrTmhIF7HIgBbnG6hm8PxLHTvaGWPkWDnEacMK9-YMvxz8zphO4eHB46kniHIOc-vop&response-X-Checksum-Sha1=f8ab68145e3a23da45be0847f114065aa2fc0f54&response-X-Checksum-Sha2=3f786b85a05e14c7ed5c24fbefbee19fbaa345e1db5d16fa9fa348c2a4e4789e < HTTP/1.1 407 Proxy Authentication Required @ 9.761 ms

backing up the Nexus configuration

Using the built-in procedure is risky - to say the least. I was unable to make it work (the restore part, I mean).

You can do this (see https://stackoverflow.com/a/45986526/651288 ):

java -jar /opt/sonatype/nexus/lib/support/nexus-orient-console.jar

CONNECT create database plocal:/home/centos/nexus310/sonatype-work/nexus3/db/component admin admin

#to export, use this:
export database component-export



#to import, use this:
drop database
create database plocal:/home/centos/nexus310/sonatype-work/nexus3/db/component
import database component-export.json.gz

I also do this:

gunzip component-export.json.gz
python -m json.tool component-export.json > component-exportPRETTY.json


and the JSON file is quite readable, probably you can even stick it into a Git repo and relax.

just replace "component" with "config" and do all over again, to backup the configuration

From an operational point of view, Nexus 3 is simply too fragile.

Nexus Groovy scripting

Huge issue in Nexus is that you can't export/import your Users/Groups/Privileges and Repository configuration. All you can do is to take a "backup", which ends of in a folder as a completely unreadable/unversionable format.

So I was looking for

when you run a Groovy script in Nexus, you have available a predefined variable "repository", which is of type org.sonatype.nexus.script.plugin.internal.provisioning.RepositoryApiImpl

This in turn contains a reference to a blobStoreManager org.sonatype.nexus.blobstore.api.BlobStoreManager and to a repositoryManager org.sonatype.nexus.repository.manager.RepositoryManager , plus a series of convenience methods to create commonly used repository formats (you are not expected to create anything fancy, most properties are assigned by default) and groups.

Key element is a org.sonatype.nexus.repository.config.Configuration object, again very disappointing since the configuration is represented by a "attribute" map (String, Map(String, Object)) which is really a stupid idea, too generic interface.

RepositoryManager has a Iterable<Repository> browse(); which returns a collection of org.sonatype.nexus.repository.Repository

Sample script:

import org.sonatype.nexus.repository.Repository

repository.repositoryManager.browse().each { Repository repo ->
    log.info("Repository: $repo")
    log.info("Repository Configuration: $repo.configuration")
}

this dumps in nexus.log the following content:

Repository: RepositoryImpl$$EnhancerByGuice$$c5f0822b{type=proxy, format=nuget, name='nuget.org-proxy'}
Repository Configuration: Configuration{repositoryName='nuget.org-proxy', recipeName='nuget-proxy', attributes={proxy={strictContentTypeValidation=true, contentMaxAge=1440, remoteUrl=https://www.nuget.org/api/v2/, metadataMaxAge=1440}, negativeCache={}, storage={blobStoreName=default}, nugetProxy={}, httpclient={connection={blocked=false, autoBlock=true}}}}

Repository: RepositoryImpl$$EnhancerByGuice$$c5f0822b{type=hosted, format=maven2, name='maven-releases'}
Repository Configuration: Configuration{repositoryName='maven-releases', recipeName='maven2-hosted', attributes={maven={versionPolicy=RELEASE, layoutPolicy=STRICT}, storage={writePolicy=ALLOW_ONCE, strictContentTypeValidation=false, blobStoreName=default}}}

Repository: RepositoryImpl$$EnhancerByGuice$$c5f0822b{type=hosted, format=maven2, name='maven-snapshots'}
Repository Configuration: Configuration{repositoryName='maven-snapshots', recipeName='maven2-hosted', attributes={maven={versionPolicy=SNAPSHOT, layoutPolicy=STRICT}, storage={writePolicy=ALLOW, strictContentTypeValidation=false, blobStoreName=default}}}

Repository: RepositoryImpl$$EnhancerByGuice$$c5f0822b{type=proxy, format=maven2, name='maven-central'}
Repository Configuration: Configuration{repositoryName='maven-central', recipeName='maven2-proxy', attributes={proxy={contentMaxAge=-1, remoteUrl=https://repo1.maven.org/maven2/, metadataMaxAge=1440}, negativeCache={timeToLive=1440, enabled=true}, storage={strictContentTypeValidation=false, blobStoreName=default}, maven-indexer={}, httpclient={connection={blocked=false, autoBlock=true}}, maven={versionPolicy=RELEASE, layoutPolicy=PERMISSIVE}}}

Repository: RepositoryImpl$$EnhancerByGuice$$c5f0822b{type=group, format=nuget, name='nuget-group'}
Repository Configuration: Configuration{repositoryName='nuget-group', recipeName='nuget-group', attributes={storage={blobStoreName=default}, nugetProxy={}, httpclient={}, group={memberNames=[nuget-hosted, nuget.org-proxy]}}}

Repository: RepositoryImpl$$EnhancerByGuice$$c5f0822b{type=hosted, format=nuget, name='nuget-hosted'}
Repository Configuration: Configuration{repositoryName='nuget-hosted', recipeName='nuget-hosted', attributes={storage={writePolicy=ALLOW, blobStoreName=default}}}

Repository: RepositoryImpl$$EnhancerByGuice$$c5f0822b{type=group, format=maven2, name='maven-public'}
Repository Configuration: Configuration{repositoryName='maven-public', recipeName='maven2-group', attributes={maven={versionPolicy=MIXED}, group={memberNames=[maven-releases, maven-snapshots, maven-central]}, storage={blobStoreName=default}}}

which is pretty good result, at least you can capture in one go all the configuration of all your repositories.



This task will delete all your repos:

import org.sonatype.nexus.repository.Repository

repository.repositoryManager.browse().each { Repository repo ->
    log.info("DELETE Repository: $repo")
    repository.repositoryManager.delete("$repo.name")
}

All the predefined variables are

core which is a org.sonatype.nexus.internal.provisioning.CoreApiImpl

repository which is a org.sonatype.nexus.script.plugin.internal.provisioning.RepositoryApiImpl

blobStore which is a org.sonatype.nexus.internal.provisioning.BlobStoreApiImpl

createFileBlobStore(final String name, final String path)
org.sonatype.nexus.blobstore.api.BlobStoreManager blobStoreManager 

security which is a org.sonatype.nexus.security.internal.SecurityApiImpl

User addUser(final String id, final String firstName, final String lastName, final String email, final boolean active,
final String password, final List roleIds)
Role addRole(final String id, final String name, final String description, final List privileges,
final List roles)
User setUserRoles(final String userId, final List roleIds)

If you do security.getSecuritySystem() you get an instance of this:
https://github.com/sonatype/nexus-public/blob/master/components/nexus-security/src/main/java/org/sonatype/nexus/security/SecuritySystem.java


Good is that if you clone the github repo nexus-book-examples you can directly open the APIs in file:///home/centos/gitclones/nexus-book-examples/scripting/apidocs/index.html


List all users with their roles

import groovy.json.JsonOutput
users = security.getSecuritySystem().listUsers()
userjson = JsonOutput.toJson(users)
log.info("USERS $userjson")

or also

import org.sonatype.nexus.security.user.UserSearchCriteria

users = security.getSecuritySystem().searchUsers(new UserSearchCriteria())
log.info("users=" + users)

Add users and roles:

privileges = [
            "nx-search-read",
            "nx-repository-view-*-*-read",
            "nx-repository-view-*-*-browse",
            "nx-repository-view-*-*-add",
            "nx-repository-view-*-*-edit",
"nx-apikey-all"]

security.addRole("deployer", "deployer", "deployment on all repositories", privileges, [])

security.addUser(userName, firstName, lastName, email, true, password, ["deployer"])

creating blobstores:

def list = ["dockerGroup", "mynpm", "pippo", "ivy", "dockerhosted", "dockerProxy", "jcenter", "pythonProxy"]
for (item in list) {
    log.info("creating blobstore " + item)
    blobStore.createFileBlobStore(item, item)
}

creating hosted repositories

import org.sonatype.nexus.repository.storage.WritePolicy;
import org.sonatype.nexus.repository.maven.VersionPolicy;
import org.sonatype.nexus.repository.storage.WritePolicy
import org.sonatype.nexus.repository.maven.LayoutPolicy


repository.createDockerHosted(name = 'pippo', httpPort = 8123, httpsPort = null, blobStoreName = 'docker', strictContentTypeValidation=true, v1Enabled=true, writePolicy = WritePolicy.ALLOW, forceBasicAuth=false)

repository.createMavenHosted(name = 'ivyhosted', blobStoreName = 'ivy', strictContentTypeValidation = true, versionPolicy = VersionPolicy.RELEASE, writePolicy= WritePolicy.ALLOW_ONCE, layoutPolicy=LayoutPolicy.PERMISSIVE )

You cannot delete anonymous, the only way is to update it:

import org.sonatype.nexus.security.user.*
import org.sonatype.nexus.security.role.*

// the following 6 lines are not required
anonymous = security.getSecuritySystem().getUser("anonymous", "default")
log.info("Anonymous before=" + anonymous)
Set allRoles = security.getSecuritySystem().listRoles();
log.info("allRoles=" + allRoles)
advRole = allRoles.find{it.roleId=='ADVRole'}
log.info("advRole=" + advRole)

// these 2 lines below do the job
advRoleIdentifier = new RoleIdentifier('default', 'ADVRole');
security.getSecuritySystem().setUsersRoles("anonymous", "default", [advRoleIdentifier].toSet())

Script to create/update a role (without affecting existing users using that role, if existing)

import org.sonatype.nexus.security.user.*
import org.sonatype.nexus.security.role.*
import static org.sonatype.nexus.security.user.UserManager.DEFAULT_SOURCE

privileges = [
            "nx-search-read",
            "nx-repository-view-*-*-read",
            "nx-repository-view-*-*-browse",
            "nx-repository-view-*-*-add",
            "nx-repository-view-*-*-edit",
            "nx-apikey-all"
]

createOrUpdateRole("pippo", privileges, [])
createOrUpdateUser("pippouser", [ "pippo", "nx-admin" ])

def createOrUpdateRole(rolename, privileges, roles) {
    log.info("calling createOrUpdateRole with parameters rolename=" + rolename + " privileges=" + privileges + " roles=" + roles)
 Set allRoles = security.getSecuritySystem().listRoles()
 Role role = allRoles.find{it.roleId==rolename}
 if (role != null) {
     log.info("existing role=" + role)
  role.setPrivileges(privileges.toSet())
  role.setRoles(roles.toSet())
  log.info("updating role=" + role)
  // security.securitySystem.getAuthorizationManager(DEFAULT_SOURCE).deleteRole(role.roleId)
  security.securitySystem.getAuthorizationManager(DEFAULT_SOURCE).updateRole(role)
  log.info("role updated=" + role)
 }
 else {
  log.info("adding role " + rolename)
  security.addRole(rolename, rolename, rolename, privileges, roles)
  log.info("role " + rolename + " successfully added")
 }
}



def createOrUpdateUser(username, roles) {
 log.info("calling createOrUpdateUser with parameters username=" + username + " roles=" + roles)
 Set allUsers = security.getSecuritySystem().listUsers()
 User user = allUsers.find{it.userId == username}
 if (user != null) {
  log.info("updating existing user=" + user)
  Set roleIdentifiers = new HashSet()
  roles.each{ role -> roleIdentifiers.add(new RoleIdentifier("default", role))}
  security.getSecuritySystem().setUsersRoles(username, "default", roleIdentifiers)
 }
 else {
  security.addUser(username, username, username, username + "@gmail.com", true, username, roles)
 }
}

Nexus programmatically create Repositories

Nexus API sucks - big time.

This https://blog.sonatype.com/automated-setup-of-a-repository-manager is a desperate attempt to automate stuff.


git clone https://github.com/sonatype/nexus-book-examples.git
cd ./nexus-book-examples/scripting/complex-script

docker run -d -p 8081:8081 --name nexus sonatype/nexus3:3.10.0
see https://hub.docker.com/r/sonatype/nexus3/tags/

check the logs by
docker exec -ti nexus /bin/bash
more /opt/sonatype/sonatype-work/nexus3/log/nexus.log


rm -rf /home/centos/.groovy/grapes/
rm -rf /home/centos/.m2/repository/



You can easily test your scripts from the Nexus UI (create execute script UI) , and also debut them remotely from Intellij :

https://support.sonatype.com/hc/en-us/articles/115015812727-Nexus-3-Groovy-Script-development-environment-setup

Try running this script:

repository.createMavenHosted('pippo')

it can't be easier!


The RepositoryAPI is available here

https://github.com/sonatype/nexus-public/blob/master/plugins/nexus-script-plugin/src/main/java/org/sonatype/nexus/script/plugin/RepositoryApi.java

Repository createMavenHosted(final String name,
final String blobStoreName,
final boolean strictContentTypeValidation,
final VersionPolicy versionPolicy,
final WritePolicy writePolicy,
final LayoutPolicy layoutPolicy);

example:


repository.createMavenHosted('private-again', 'default', true, org.sonatype.nexus.repository.maven.policy.VersionPolicy.SNAPSHOT, org.sonatype.nexus.repository.storage.WritePolicy.ALLOW_ONCE, org.sonatype.nexus.repository.maven.LayoutPolicy.STRICT)



To list repositories (unfortunately it returns only partial information about the Repository):

curl -X GET --header 'Accept: application/json' 'http://localhost:8081/service/rest/beta/repositories'

[ {
  "name" : "nuget.org-proxy",
  "format" : "nuget",
  "type" : "proxy",
  "url" : "http://localhost:8081/repository/nuget.org-proxy"
}, {
  "name" : "maven-releases",
  "format" : "maven2",
  "type" : "hosted",
  "url" : "http://localhost:8081/repository/maven-releases"
}, {
  "name" : "maven-snapshots",
  "format" : "maven2",
  "type" : "hosted",
  "url" : "http://localhost:8081/repository/maven-snapshots"
}, {
  "name" : "maven-central",
  "format" : "maven2",
  "type" : "proxy",
  "url" : "http://localhost:8081/repository/maven-central"
}, {
  "name" : "nuget-group",
  "format" : "nuget",
  "type" : "group",
  "url" : "http://localhost:8081/repository/nuget-group"
}, {
  "name" : "nuget-hosted",
  "format" : "nuget",
  "type" : "hosted",
  "url" : "http://localhost:8081/repository/nuget-hosted"
}, {
  "name" : "maven-public",
  "format" : "maven2",
  "type" : "group",
  "url" : "http://localhost:8081/repository/maven-public"
} ]

Ref: https://help.sonatype.com/repomanager3/rest-and-integration-api/script-api/writing-scripts about Groovy Scripting in Nexus

Nexus and Groovy for Setup Automation

Amazingly few people automate their Nexus administration. I guess the fault lies mostly in the company behind Nexus, who does very little to make their API usable and well documented.

This post https://blog.soebes.de/blog/2017/10/02/nexus-scripted-setup/ made me discover this API:

https://github.com/sonatype/nexus-public/blob/master/plugins/nexus-script-plugin/src/main/java/org/sonatype/nexus/script/plugin/RepositoryApi.java

Very good reading also here https://support.sonatype.com/hc/en-us/articles/115015812727-Nexus-3-Groovy-Script-development-environment-setup about using the Nexus book examples to automate the execution of these Groovy/Java scripts.

References http://www.javamonamour.org/2018/03/nexus-repository-administration.html on same topic of Automation (via REST api)

Nexus Maven repository migration

I need to migrate a Nexus 2.14 Maven repository to Nexus 3.10.

Unfortunately the Nexus product doesn't care much about migrations... probably they believe that, once installed, their product is immovable, like a sick elephant.

With a Nexus 2.14 to Nexus 2.14 migration, the repository structure is so simple that you can do it with a rsync.
But Nexus 3.10 is a totally different beast.

Luckily there are people who wrote tools:


https://github.com/simpligility/maven-repository-tools



I have to migrate from http://localhost:8081/nexus/content/repositories/releases/ (Nexus 2.14) to http://localhost:8181/repository/releases/ (Nexus 3.10)

git clone https://github.com/simpligility/maven-repository-tools.git
cd maven-repository-tools
mvn package
java -jar maven-repository-provisioner/target/maven-repository-provisioner-1.3.2-SNAPSHOT-jar-with-dependencies.jar

java -jar maven-repository-provisioner/target/maven-repository-provisioner-1.3.2-SNAPSHOT-jar-with-dependencies.jar -s http://localhost:8081/nexus/content/repositories/releases/ -t http://localhost:8181/repository/releases/ -u admin -p admin123 -a org.pierre:pvjoinfacestest:war:1.0


tragically, the tool doesn't allow wildcards.... so you must know in advance the gav of ALL the artifacts you have to migrate.... my impression is also that the .war is not really uploaded!


I am trying this one https://github.com/DarthHater/nexus-repository-import-scripts

copy https://github.com/DarthHater/nexus-repository-import-scripts/blob/master/mavenimport.sh to my /home/centos/nexus214/sonatype-work/nexus/storage/releases folder, then run

./mavenimport.sh -r http://localhost:8181/repository/releases/ -u admin -p admin123

and it works!

Nexus HTTP/1.1 407 Proxy Authentication Required

While trying to connect to https://repo.typesafe.com/typesafe/ivy-releases/ and https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/ over a corporate proxy with these 2 sites whitelisted, we keep getting:


org.sonatype.nexus.repository.maven.internal.proxy.MavenProxyFacet - Exception org.sonatype.nexus.repository.proxy.ProxyServiceException: HTTP/1.1 407 Proxy Authentication Required checking remote for update, proxy repo ivyreleases_proxy failed to fetch com.typesafe.sbt/sbt-native-packager/scala_2.12/sbt_1.0/1.3.4/ivys/ivy.xml with status line HTTP/1.1 407 Proxy Authentication Required, content not in cache.



If you enable ROOT logging at DEBUG (menu Support/Logging in Nexus console) you will see that a redirect is taking place, but the LAZY Nexus developer only logs the message and not the additional ESSENTIAL parameters to understand what is going on https://github.com/sonatype/nexus-public/blob/master/components/nexus-repository/src/main/java/org/sonatype/nexus/repository/proxy/ProxyFacetSupport.java, that is a redirection to https://dl.bintray.com

2018-09-03 13:11:48,044+0200 DEBUG [qtp46263989-6144]  *UNKNOWN org.sonatype.nexus.repository.maven.internal.proxy.MavenProxyFacet - Fetching: GET https://repo.typesafe.com/typesafe/ivy-releases/com.typesafe.sbt/sbt-native-packager/scala_2.12/sbt_1.0/1.3.4/ivys/ivy.xml HTTP/1.1
2018-09-03 13:11:48,046+0200 DEBUG [qtp46263989-6144]  *UNKNOWN org.sonatype.nexus.httpclient.outbound - https://repo.typesafe.com/typesafe/ivy-releases/com.typesafe.sbt/sbt-native-packager/scala_2.12/sbt_1.0/1.3.4/ivys/ivy.xml > GET /typesafe/ivy-releases/com.typesafe.sbt/sbt-native-packager/scala_2.12/sbt_1.0/1.3.4/ivys/ivy.xml HTTP/1.1
2018-09-03 13:11:48,046+0200 DEBUG [qtp46263989-6144]  *UNKNOWN org.sonatype.nexus.internal.httpclient.SharedHttpClientConnectionManager - Connection request: [route: {tls}->http://ourproxyserver:8080->https://repo.typesafe.com:443][total kept alive: 0; route allocated: 0 of 20; total allocated: 0 of 200]
2018-09-03 13:11:48,046+0200 DEBUG [qtp46263989-6144]  *UNKNOWN org.sonatype.nexus.internal.httpclient.SharedHttpClientConnectionManager - Connection leased: [id: 478][route: {tls}->http://ourproxyserver:8080->https://repo.typesafe.com:443][total kept alive: 0; route allocated: 1 of 20; total allocated: 1 of 200]
2018-09-03 13:11:48,448+0200 DEBUG [qtp46263989-6144]  *UNKNOWN org.sonatype.nexus.httpclient.outbound - https://repo.typesafe.com/typesafe/ivy-releases/com.typesafe.sbt/sbt-native-packager/scala_2.12/sbt_1.0/1.3.4/ivys/ivy.xml < HTTP/1.1 302 Moved Temporarily @ 402.5 ms
2018-09-03 13:11:48,448+0200 DEBUG [qtp46263989-6144]  *UNKNOWN org.sonatype.nexus.internal.httpclient.SharedHttpClientConnectionManager - Connection [id: 478][route: {tls}->http://ourproxyserver:8080->https://repo.typesafe.com:443] can be kept alive for 30.0 seconds
2018-09-03 13:11:48,449+0200 DEBUG [qtp46263989-6144]  *UNKNOWN org.sonatype.nexus.internal.httpclient.SharedHttpClientConnectionManager - Connection released: [id: 478][route: {tls}->http://ourproxyserver:8080->https://repo.typesafe.com:443][total kept alive: 1; route allocated: 1 of 20; total allocated: 1 of 200]
2018-09-03 13:11:48,449+0200 DEBUG [qtp46263989-6144]  *UNKNOWN org.sonatype.nexus.httpclient.outbound - https://dl.bintray.com/typesafe/ivy-releases/com.typesafe.sbt/sbt-native-packager/scala_2.12/sbt_1.0/1.3.4/ivys/ivy.xml > GET /typesafe/ivy-releases/com.typesafe.sbt/sbt-native-packager/scala_2.12/sbt_1.0/1.3.4/ivys/ivy.xml HTTP/1.1
2018-09-03 13:11:48,449+0200 DEBUG [qtp46263989-6144]  *UNKNOWN org.sonatype.nexus.internal.httpclient.SharedHttpClientConnectionManager - Connection request: [route: {tls}->http://ourproxyserver:8080->https://dl.bintray.com:443][total kept alive: 1; route allocated: 0 of 20; total allocated: 1 of 200]
2018-09-03 13:11:48,449+0200 DEBUG [qtp46263989-6144]  *UNKNOWN org.sonatype.nexus.internal.httpclient.SharedHttpClientConnectionManager - Connection leased: [id: 479][route: {tls}->http://ourproxyserver:8080->https://dl.bintray.com:443][total kept alive: 1; route allocated: 1 of 20; total allocated: 2 of 200]
2018-09-03 13:11:48,466+0200 DEBUG [qtp46263989-6144]  *UNKNOWN org.sonatype.nexus.internal.httpclient.SharedHttpClientConnectionManager - Connection released: [id: 479][route: {tls}->http://ourproxyserver:8080->https://dl.bintray.com:443][total kept alive: 1; route allocated: 0 of 20; total allocated: 1 of 200]
2018-09-03 13:11:48,466+0200 DEBUG [qtp46263989-6144]  *UNKNOWN org.sonatype.nexus.httpclient.outbound - https://dl.bintray.com/typesafe/ivy-releases/com.typesafe.sbt/sbt-native-packager/scala_2.12/sbt_1.0/1.3.4/ivys/ivy.xml < HTTP/1.1 407 Proxy Authentication Required @ 17.12 ms
2018-09-03 13:11:48,466+0200 DEBUG [qtp46263989-6144]  *UNKNOWN org.sonatype.nexus.repository.maven.internal.proxy.MavenProxyFacet - Response: HttpResponseProxy{HTTP/1.1 407 Proxy Authentication Required [Proxy-Authenticate: NTLM, Cache-Control: no-cache, Pragma: no-cache, Content-Type: text/html; charset=utf-8, Proxy-Connection: close, Connection: close, Content-Length: 850] org.apache.http.entity.BufferedHttpEntity@3326370a}
2018-09-03 13:11:48,466+0200 DEBUG [qtp46263989-6144]  *UNKNOWN org.sonatype.nexus.repository.maven.internal.proxy.MavenProxyFacet - Status: HTTP/1.1 407 Proxy Authentication Required
2018-09-03 13:11:48,467+0200 WARN  [qtp46263989-6144]  *UNKNOWN org.sonatype.nexus.repository.maven.internal.proxy.MavenProxyFacet - Exception org.sonatype.nexus.repository.proxy.ProxyServiceException: HTTP/1.1 407 Proxy Authentication Required checking remote for update, proxy repo ivyreleases_proxy failed to fetch com.typesafe.sbt/sbt-native-packager/scala_2.12/sbt_1.0/1.3.4/ivys/ivy.xml with status line HTTP/1.1 407 Proxy Authentication Required, content not in cache.
org.sonatype.nexus.repository.proxy.ProxyServiceException: HTTP/1.1 407 Proxy Authentication Required
 at org.sonatype.nexus.repository.proxy.ProxyFacetSupport.mayThrowProxyServiceException(ProxyFacetSupport.java:426)
 at org.sonatype.nexus.repository.proxy.ProxyFacetSupport.fetch(ProxyFacetSupport.java:400)
 at org.sonatype.nexus.repository.proxy.ProxyFacetSupport.fetch(ProxyFacetSupport.java:346)
 at org.sonatype.nexus.repository.maven.internal.proxy.MavenProxyFacet.fetch(MavenProxyFacet.java:73)
 at org.sonatype.nexus.repository.proxy.ProxyFacetSupport.doGet(ProxyFacetSupport.java:220)
 at org.sonatype.nexus.repository.proxy.ProxyFacetSupport.lambda$1(ProxyFacetSupport.java:209)
 at org.sonatype.nexus.repository.proxy.ProxyFacetSupport$$Lambda$269/1592429552.call(Unknown Source)
 at org.sonatype.nexus.repository.proxy.Cooperation$CooperatingFuture.download(Cooperation.java:262)
 at org.sonatype.nexus.repository.proxy.Cooperation.download(Cooperation.java:194)
 at org.sonatype.nexus.repository.proxy.Cooperation.cooperate(Cooperation.java:93)
 at org.sonatype.nexus.repository.proxy.ProxyFacetSupport.get(ProxyFacetSupport.java:201)
 at org.sonatype.nexus.repository.proxy.ProxyHandler.handle(ProxyHandler.java:49)
 at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
 at org.sonatype.nexus.repository.storage.UnitOfWorkHandler.handle(UnitOfWorkHandler.java:39)
 at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
 at org.sonatype.nexus.repository.view.handlers.ContentHeadersHandler.handle(ContentHeadersHandler.java:44)
 at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
 at org.sonatype.nexus.repository.maven.internal.VersionPolicyHandler.handle(VersionPolicyHandler.java:61)
 at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
 at org.sonatype.nexus.repository.http.PartialFetchHandler.handle(PartialFetchHandler.java:55)
 at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
 at org.sonatype.nexus.repository.cache.NegativeCacheHandler.handle(NegativeCacheHandler.java:42)
 at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
 at org.sonatype.nexus.repository.view.handlers.ConditionalRequestHandler.handle(ConditionalRequestHandler.java:72)
 at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
 at org.sonatype.nexus.repository.assetdownloadcount.internal.AssetDownloadCountContributedHandler.handle(AssetDownloadCountContributedHandler.java:53)
 at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
 at com.sonatype.nexus.clm.internal.QuarantineContributedHandler.handle(QuarantineContributedHandler.java:68)
 at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
 at com.sonatype.nexus.usertoken.plugin.internal.UserTokenHandler.handle(UserTokenHandler.java:70)
 at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
 at org.sonatype.nexus.repository.view.handlers.HandlerContributor.handle(HandlerContributor.java:67)
 at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
 at org.sonatype.nexus.repository.view.handlers.ExceptionHandler.handle(ExceptionHandler.java:44)
 at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
 at org.sonatype.nexus.repository.security.SecurityHandler.handle(SecurityHandler.java:52)
 at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
 at org.sonatype.nexus.repository.view.handlers.TimingHandler.handle(TimingHandler.java:46)
 at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
 at org.sonatype.nexus.repository.view.Context.start(Context.java:114)
 at org.sonatype.nexus.repository.view.Router.dispatch(Router.java:64)
 at org.sonatype.nexus.repository.view.ConfigurableViewFacet.dispatch(ConfigurableViewFacet.java:52)
 at org.sonatype.nexus.repository.view.ConfigurableViewFacet.dispatch(ConfigurableViewFacet.java:43)
 at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.dispatchAndSend(ViewServlet.java:211)
 at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.doService(ViewServlet.java:173)
 at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.service(ViewServlet.java:126)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
 at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286)
 at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276)
 at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181)
 at com.google.inject.servlet.DynamicServletPipeline.service(DynamicServletPipeline.java:71)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
 at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:112)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
 at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
 at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
 at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
 at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
 at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
 at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
 at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
 at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
 at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
 at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
 at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
 at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
 at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
 at org.sonatype.nexus.security.SecurityFilter.executeChain(SecurityFilter.java:85)
 at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
 at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
 at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
 at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
 at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
 at org.sonatype.nexus.security.SecurityFilter.doFilterInternal(SecurityFilter.java:101)
 at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at org.sonatype.nexus.repository.httpbridge.internal.ExhaustRequestFilter.doFilter(ExhaustRequestFilter.java:71)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at com.sonatype.nexus.licensing.internal.LicensingRedirectFilter.doFilter(LicensingRedirectFilter.java:108)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(AbstractInstrumentedFilter.java:97)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at org.sonatype.nexus.internal.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:68)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at org.sonatype.nexus.internal.web.EnvironmentFilter.doFilter(EnvironmentFilter.java:102)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at org.sonatype.nexus.internal.web.HeaderPatternFilter.doFilter(HeaderPatternFilter.java:98)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at com.google.inject.servlet.DynamicFilterPipeline.dispatch(DynamicFilterPipeline.java:104)
 at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:135)
 at org.sonatype.nexus.bootstrap.osgi.DelegatingFilter.doFilter(DelegatingFilter.java:73)
 at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)
 at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
 at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
 at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
 at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
 at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
 at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
 at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
 at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
 at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
 at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
 at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
 at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
 at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
 at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
 at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
 at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:175)
 at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
 at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
 at org.eclipse.jetty.server.Server.handle(Server.java:530)
 at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
 at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
 at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
 at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
 at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
 at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
 at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
 at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
 at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
 at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
 at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
 at java.lang.Thread.run(Thread.java:745)

For pypi.org , you might need to whitelist also https://files.pythonhosted.org:443

Nexus 3.10 as Python pypi proxy

sudo yum -y update
sudo yum -y install python-pip
pip --help
pip -V
mkdir ~/.pip
cd .pip/
vi pip.conf

enter this:

[global]
index = http://localhost:8181/repository/pypi/
index-url = http://localhost:8181/repository/pypi/simple

Create a nexus Pypi proxy repo "pypi" pointing to https://pypi.org/ . Leave all the defaults as they are.

pip install lino-amici

this should populate your nexus:

Looking in indexes: http://localhost:8181/repository/pypi/simple
Collecting lino-amici
  Downloading http://localhost:8181/repository/pypi/packages/eb/c3/7bdd189fd446effe7ea61d2136dfaec478ec7c5b84f694c6b6e43656f319/lino-amici-18.8.0.tar.gz
Collecting lino-xl (from lino-amici)
  Downloading http://localhost:8181/repository/pypi/packages/02/34/6ab429aac3f80aa64fd00c8f3dd76a9ca9ecf25d7e03f1e48981c7031542/lino-xl-18.4.0.tar.gz (2.3MB)
    100% |████████████████████████████████| 2.3MB 92.2MB/s 
Collecting vobject (from lino-amici)
  Downloading http://localhost:8181/repository/pypi/packages/da/ce/27c48c0e39cc69ffe7f6e3751734f6073539bf18a0cfe564e973a3709a52/vobject-0.9.6.1.tar.gz (58kB)
    100% |████████████████████████████████| 61kB 19.8MB/s 
Collecting lino (from lino-xl->lino-amici)
  Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPConnectionPool(host='localhost', port=8181): Read timed out. (read timeout=15)",)': /repository/pypi/packages/68/9e/39e5e96b9b8273f0f29943cddd51873ec670657ccf9918727d97ca6628e2/lino-18.8.0.tar.gz
  Downloading http://localhost:8181/repository/pypi/packages/68/9e/39e5e96b9b8273f0f29943cddd51873ec670657ccf9918727d97ca6628e2/lino-18.8.0.tar.gz (10.6MB)
    100% |████████████████████████████████| 10.6MB 89.8MB/s 
Collecting odfpy (from lino-xl->lino-amici)
  Downloading http://localhost:8181/repository/pypi/packages/01/0f/c9971c99d0d06024a1652f467427ff3f1a1136237e5740da715c5b208a48/odfpy-1.3.6.tar.gz (691kB)
    100% |████████████████████████████████| 696kB 42.6MB/s 
Collecting bleach (from lino-xl->lino-amici)
  Downloading http://localhost:8181/repository/pypi/packages/94/aa/0f7ce53f8688bb9f80c0cffacc3964ddfe08321c509c0bfe5062848951f9/bleach-2.1.4-py2.py3-none-any.whl
Collecting weasyprint (from lino-xl->lino-amici)
  Downloading http://localhost:8181/repository/pypi/packages/7e/4c/cf2ec7abf7f84a2d1325d01dcac1d4bcb77f41117101fe564eb76952c65f/WeasyPrint-0.42.3.tar.gz (399kB)
    100% |████████████████████████████████| 409kB 37.7MB/s 
    Complete output from command python setup.py egg_info:
    error in WeasyPrint setup command: Invalid environment marker: python_version < "3.0"
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-sPdLGr/weasyprint/

No worries about the message about the python version.... I have 2.7.... not willing to upgrade for now...

Gradle and Nexus

download Gradle binaries

mkdir /home/centos/gradle
unzip -d /home/centos/gradle/ gradle-4.8-bin.zip
export PATH=$PATH:/home/centos/gradle/gradle-4.8/bin/
gradle -version
cd gitclones/
git clone https://github.com/buzdin/allure-junit5-course

cd allure-junit5-course/

vi build.gradle

replace

repositories {
jcenter()
}

with

repositories {
maven {
url "http://localhost:8181/repository/maven-public/"
}
}


if your "group" maven-public repository points only to Maven Central, the build will fail

rm -rf .gradle/
gradle build --refresh-dependencies --debug



> Could not resolve all artifacts for configuration ':classpath'.
> Could not find io.qameta.allure:allure-gradle:2.4.
Searched in the following locations:
- http://localhost:8181/repository/maven-public/io/qameta/allure/allure-gradle/2.4/allure-gradle-2.4.pom
- http://localhost:8181/repository/maven-public/io/qameta/allure/allure-gradle/2.4/allure-gradle-2.4.jar
Required by:
project :


If you add to the group also a Proxy Repo "jcenter" pointing to https://jcenter.bintray.com/, everything works

Nexus proxy to npm registry

https://nodejs.org/en/
https://www.digitalocean.com/community/tutorials/how-to-install-node-js-on-a-centos-7-server


Install node.js

cd ~
wget http://nodejs.org/dist/v0.10.30/node-v0.10.30-linux-x64.tar.gz
sudo tar --strip-components 1 -xzvf node-v* -C /usr/local
node --version
npm

https://help.sonatype.com/repomanager3/node-packaged-modules-and-npm-registries

In nexus 3, create a Proxy NPM repo pointing to https://registry.npmjs.org and called npm-all


npm config set registry http://localhost:8081/repository/npm-all/
npm install --save-dev babel-loader babel-core babel-preset-env webpack


it works! I only get this error

engine p-try@1.0.0: wanted: {"node":">=4"} (current: {"node":"0.10.30","npm":"1.4.21"})
npm ERR! Error: Package 'webassemblyjs-ast' not found : repository/npm-all/webassemblyjs/ast
npm ERR!     at RegClient. (/usr/local/lib/node_modules/npm/node_modules/npm-registry-client/lib/request.js:300:14)
npm ERR!     at Request._callback (/usr/local/lib/node_modules/npm/node_modules/npm-registry-client/lib/request.js:238:65)
npm ERR!     at Request.self.callback (/usr/local/lib/node_modules/npm/node_modules/request/request.js:123:22)
npm ERR!     at Request.emit (events.js:98:17)
npm ERR!     at Request. (/usr/local/lib/node_modules/npm/node_modules/request/request.js:893:14)
npm ERR!     at Request.emit (events.js:117:20)
npm ERR!     at IncomingMessage. (/usr/local/lib/node_modules/npm/node_modules/request/request.js:844:12)
npm ERR!     at IncomingMessage.emit (events.js:117:20)
npm ERR!     at _stream_readable.js:938:16
npm ERR!     at process._tickCallback (node.js:419:13)
npm ERR! If you need help, you may report this *entire* log,
npm ERR! including the npm and node versions, at:
npm ERR!     

to clean npm local repository, use "npm cache clean"

Sanity check on a Nexus 2 Maven Proxy repo

The task is: verify that all the JARs in a Nexus 2 repo are still in Maven Central.

I did a
find /path/to/nexusdata/central -name "*.jar" > alljarsfiltered.txt


and then run this Java application:

package mavenchecker;

import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.stream.Stream;

public class MavenChecker {
 public static ArrayList<String> errors = new ArrayList<String>();
 public static ArrayList<String> nonexisting = new ArrayList<String>();
 static int count  = 0;

 public static void main(String[] args) {

  String fileName = "/home/centos/Downloads/alljarsfiltered.txt";
  
  // read file into stream, try-with-resources
  try (Stream<String> stream = Files.lines(Paths.get(fileName)).parallel()) {

   stream.forEach((line) -> checkLine(line));
   

  } catch (IOException e) {
   e.printStackTrace();
  }

  for (String error : errors) {
   System.out.println("ERROR : " + error);
  }

  for (String nonexistingItem : nonexisting) {
   System.out.println("nonexisting : " + nonexistingItem);
  }

 }

 private static void checkLine(String line) {
  count++;
  if (count % 100 == 0) {
   System.out.println("COUNT " + count);
  }
  if (!line.endsWith(".jar"))
   return;

  String gavstring = line.replace("/path/to/nexusdata/central/", "");
  String[] parts = gavstring.split("/");
  int len = parts.length;
  if (len < 4) {
   System.err.println("invalid length for gavstring " + gavstring + " len = " + len + " should be at least 4");
   errors.add(gavstring);
  } else {
   String filename = parts[len - 1];
   String version = parts[len - 2];
   String artifactid = parts[len - 3];
   String group = String.join(".", Arrays.copyOfRange(parts, 0, len - 3));
   String message = "filename=" + filename + " group=" + group + " artifactid=" + artifactid + " version=" + version;
   //System.out.println(message);
   boolean exists = exists("https://repo.maven.apache.org/maven2/" + gavstring);
   if (!exists) nonexisting.add(gavstring);
   
   
  }

 }
 
 public static boolean exists(String URLName){
     try {
       HttpURLConnection.setFollowRedirects(false);
       // note : you may also need
       //        HttpURLConnection.setInstanceFollowRedirects(false)
       HttpURLConnection con =
          (HttpURLConnection) new URL(URLName).openConnection();
       con.setRequestMethod("HEAD");
       return (con.getResponseCode() == HttpURLConnection.HTTP_OK);
     }
     catch (Exception e) {
        e.printStackTrace();
        return false;
     }
   }

}

Nexus Firewall

http://www.javamonamour.org/2017/09/software-vulnerability-control-with.html

As usual, Sonatype scatters his product documentation across the 5 Oceans, in the most disparate formats (blogs, videos, poorly formatted wiki pages etc)


https://my.sonatype.com/firewall/


Good video here https://blog.sonatype.com/nexus-firewall-for-oss-users

https://help.sonatype.com/iqserver/nexus-firewall-quick-start quick start

The Firewall product is really simple: given a GAV, it checks a DB (NIST Vulnerabilities) for all its vulnerabilities, and applies a bunch of rules to determine if the component is risky. If it's risky, it quarantines it, but provides a function to "unlock it" to the end user (Maven).