Project

General

Profile

bug #2957

running maven site:site for Taxonomic Editor fails

Added by Andreas Kohlbecker about 7 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Highest
Category:
taxeditor
Target version:
Start date:
07/17/2012
Due date:
% Done:

50%

Estimated time:
8.00 h
Severity:
major
Found in Version:
Tags:

Description

running mvn3 site:site on Taxonomic Editor fails locally and on the integration server even if executed manually (not via the jenkins job)

History

#1 Updated by Andreas Müller almost 7 years ago

  • Assignee changed from Niels Hoffmann to Andreas Kohlbecker

#2 Updated by Andreas Müller almost 7 years ago

  • Target version changed from TaxEditor RELEASE 3.0.12 to TaxEditor RELEASE 3.0.13

#3 Updated by Cherian Mathew over 6 years ago

  • Target version set to TaxEditor RELEASE 3.1.3

#4 Updated by Cherian Mathew about 6 years ago

  • Target version deleted ()

#5 Updated by Cherian Mathew over 5 years ago

  • Target version deleted (TaxEditor RELEASE 3.3.0)

#6 Updated by Andreas Kohlbecker over 5 years ago

this is broken since about 2 Years!

Is there a volunteer who wants to care for this?

Andreas

#7 Updated by Cherian Mathew over 5 years ago

  • Assignee changed from Andreas Kohlbecker to c.mathew -

#8 Updated by Cherian Mathew over 5 years ago

  • Status changed from New to In Progress

The main problem with the site goal is the fact that the 'project-info-reports:dependencies' plugin : http://maven.apache.org/plugins/maven-project-info-reports-plugin/dependencies-mojo.html is run for each sub-project.

This plugin tries to check repository locations for all the dependency jars. In the case of the cdmlib jars, this process is really slow and seems to wait for the responses from the repositories,

[WARNING] The repository url 'http://oss.sonatype.org/service/local/staging/deploy/maven2/' is invalid - Repository 'sonatype-nexus-staging' will be blacklisted.
[WARNING] The repository url 'file://${basedir}/stage' is invalid - Repository 'local-mime4j-stage-repository' will be blacklisted.
[WARNING] The repository url 'http://repository.jboss.org/maven2' is invalid - Repository 'jboss' will be blacklisted.
[WARNING] The repository url 'http://repository.springsource.com/maven/bundles/release' is invalid - Repository 'SpringSource Enterprise Bundle Repository - SpringSource Bundle Releases' will be blacklisted.
[WARNING] The repository url 'http://repository.springsource.com/maven/bundles/milestone' is invalid - Repository 'SpringSource Enterprise Bundle Repository - External Bundle Milestones' will be blacklisted.
[WARNING] The repository url 'http://repository.springsource.com/maven/bundles/external' is invalid - Repository 'SpringSource Enterprise Bundle Repository - External Bundle Releases' will be blacklisted.
[ERROR] Unable to connect to: http://people.apache.org/repo/m2-ibiblio-rsync-repository

This potentially affects all the sub-projects of the taxeditor since they are dependent an these jars.

There seems to be two ways to get around this,

  • disable the dependency location check adding '-Ddependency.locations.enabled=false' to the mvn command. This will result in the 'Dependencies' : http://cybertaxonomy.eu/taxeditor/dependencies.html page being empty.

  • fix the urls so that the build passes through quickly

#9 Updated by Andreas Kohlbecker over 5 years ago

We should fix the repository configuration in the pom

for repository.springsource.com, see http://ebr.springsource.com/repository/app/faq#q2

#10 Updated by Cherian Mathew over 5 years ago

Created a new job http://int.e-taxonomy.eu/jenkins/view/EDIT%20Taxonomic%20Editor/job/taxeditor-SITE-DEPLOY/ for site deply.

Letting the job run to its completion takes about one hour : http://int.e-taxonomy.eu/jenkins/job/taxeditor-SITE-DEPLOY/5/console and shows a vast number of repository http checks done for the project dependencies.

So the 'slowness' of the build is clearly due to the 'project-info-reports:dependencies'.

There is a separate problem at the end due to dependency resolution.

#11 Updated by Cherian Mathew over 5 years ago

The dependency problem is due to missing jars in the local repository. Running 'mvn install' before the site goals fixes this problem.

Updated the job to run 'site:deploy' and this works fine with the '-Ddependency.locations.enabled=false' option.

When running without the option the job takes more than 5 hours and had to be aborted, because it was making other routine jobs run slower than usual.

By default the option has been set.

We should probably try this again (without the option) at the time of the next release - preferably at night. If it does finish then we should probably backup the generated project-dependencies.html and reuse it in the future.

This will allow us to run the site job with the above option unless we know that the project dependency has changed.

The current status of the site job is that it has run successfully with the option for release 3.3.0 (meaning no project dependencies page).

#12 Updated by Andreas Kohlbecker over 5 years ago

So we are having two problems here:

  1. jars are missing when mvn install was not run - a minor one

  2. Problems due to inaccessible remote repositories when creating the project-dependencies.html@. This step can be skipped by @-Ddependency.locations.enabled=false

  • how many and which dependecies are exactly causing this probelm, only those listed above? Is it possible to fix them?

#13 Updated by Cherian Mathew over 5 years ago

  1. is fixed by just adding an 'mvn install' - doesn't cost much.

  2. is more tricky. I can't get much information on the net on the exact behaviour of the maven-project-info-reports-plugin (http://maven.apache.org/plugins/maven-project-info-reports-plugin/dependencies-mojo.html) but when running 'mvn -X site:site', it seems that the plugin goes through every dependency jar of every project and tries to check maven repositories for each of those jars to add to the report. Since the cdmlib and its (large number of) jars are used in every project, this takes some time and bandwidth. To fix this in entirety we will have to find all the urls of all the jars, check if they are correct and then fix them. Even if we do fix them the job will still take some amount of time and bandwidth.

#14 Updated by Andreas Kohlbecker over 5 years ago

  • Target version deleted ()

no release 3.3.1 of the taxeditor so I am moving all tickets to the next milestone 3.3.2

#15 Updated by Cherian Mathew over 4 years ago

  • Target version deleted (TaxEditor RELEASE 3.4.0)

Moving tickets to 3.4.1 milestone

#16 Updated by Andreas Kohlbecker over 4 years ago

  • Keywords set to maven

#17 Updated by Cherian Mathew over 4 years ago

  • Target version deleted ()

Moving tickets that were auto assigned to me to the unassigned milestone

#18 Updated by Andreas Kohlbecker almost 3 years ago

  • Description updated (diff)
  • Assignee deleted (Cherian Mathew)
  • Priority changed from New to Highest

This is broken since 4 Years. Who can take care for this ticket?

#19 Updated by Andreas Müller over 2 years ago

  • Assignee set to Patrick Plitzner
  • Target version changed from Unassigned CDM tickets to Release 4.5
  • Estimated time set to 8.00 h

#20 Updated by Patrick Plitzner over 2 years ago

  • Status changed from In Progress to Feedback
  • Assignee changed from Patrick Plitzner to Andreas Kohlbecker

On my local machine I tried running mvn site:site which at first crashed. Then I tried mvn install first and then it worked but took very long so I cancelled the procedure. I then tested with the --non-recursive flag because all the site .apt-files are in the root folder. This was done very quickly and all the sites were generated. Running it with this option should be fine and could be added to the jenkins job because I did not find any other .apt-files in any sub folders. Or did I miss something and does this maven goal need other information from the sub projects?

#21 Updated by Andreas Kohlbecker over 2 years ago

  • Assignee changed from Andreas Kohlbecker to Patrick Plitzner

As far as i can see there is no other information from subprojects that needs to be included into the site doc.
Adding --non-recursive to the jenkins job seems to be fine.

#22 Updated by Patrick Plitzner over 2 years ago

  • Assignee changed from Patrick Plitzner to Andreas Kohlbecker

Andreas Kohlbecker wrote:

As far as i can see there is no other information from subprojects that needs to be included into the site doc.
Adding --non-recursive to the jenkins job seems to be fine.

I added the --non-recursive flag to the jenkins job and updated the job description. Now, we have to wait for the release!

#23 Updated by Andreas Müller over 2 years ago

  • Status changed from Feedback to Resolved

#24 Updated by Andreas Müller over 2 years ago

  • % Done changed from 0 to 50

#25 Updated by Andreas Müller over 2 years ago

  • Private changed from Yes to No

#26 Updated by Andreas Kohlbecker over 2 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 40 MB)