NEW: The buildlist task can now have `root` and `leaf` subelements to specify the organisation (and other attributes) of the root and leaf modules

This commit is contained in:
Maarten Coene 2018-09-16 23:43:30 +02:00
parent 0712acabd5
commit b3f7c85969
3 changed files with 217 additions and 52 deletions

View File

@ -101,6 +101,7 @@ For details about the following changes, check our JIRA install at link:https://
- NEW: Add ivy.maven.lookup.sources and ivy.maven.lookup.javadoc variables to control the lookup of the additional artifacts. Defaults to true, for backward compatibility (jira:IVY-1529[])
- NEW: Add (conditional) support for SHA-256, SHA-512 and SHA-384 checksum algorithms (jira:IVY-1554[]) (Thanks to Jaikiran Pai)
- NEW: The standalone Ivy jar can now be used to generate a pom file for the resolved module, using the `makepom` option (Thanks to link:https://github.com/aanno[Thomas Pasch])
- NEW: The buildlist task can now have `root` and `leaf` subelements to specify the organisation (and other attributes) of the root and leaf modules (jira:/IVY-1242[]) and (jira:IVY-1293[])
////

View File

@ -32,9 +32,9 @@ When the `ivy.xml` of the modules that you want to order contains link:../ivyfil
(*__since 1.4__*) The `ivy.sorted.modules` property is set in Ant project at the end of the task with a comma separated list of ordered modules. This can be useful for debug or information purpose.
[*__since 2.0__*]
(*__since 2.0__*) The `root` and `leaf` attributes can be a delimited list of modules to use as roots. These modules, and all their dependencies will be included in the build list.
The `root` and `leaf` attributes can be a delimited list of modules to use as roots. These modules, and all their dependencies will be included in the build list.
(*__since 2.5__*) The root and leaf modules can also be specified as nested `root` and `leaf` elements. This way, not only the module name can be specified, but also the organisation, revision and branch of the root/leaf.
By default, all the modules included in a circular dependency are grouped together so that any dependency of any module in the loop will appear before the modules in the loop. This guarantees that if there is a dependency path between a module A and a module B (but no dependency path from B to A), B will always appear before A even if A is included in a loop in the provided set of modules to sort.
Note that a circular dependency can also trigger a failure depending on the value configured in the `circularDependencyStrategy` of your link:../settings/conf{outfilesuffix}#circularDependencyStrategy[settings]
@ -76,6 +76,29 @@ This field is ignored when neither root nor leaf is filled.|No. Defaults to no `
|settingsRef|(*__since 2.0__*) A reference to Ivy settings that must be used by this task|No, `ivy.instance` is taken by default.
|=======
== Child elements
[options="header",cols="15%,50%,35%"]
|=======
|Element|Description|Cardinality
|root|(*__since 2.5__*) Declares a root module.
This element takes the following attributes: +
* `organisation`: the organisation of the root module (defaults to *) +
* `module`: the name of the root module (defaults to *) +
* `revision`: the revision of the root module (defaults to *) +
* `branch`: the branch of the root module (default to *)
* `file`: a specific ivy.xml file to use as root module|0..n
|leaf|(*__since 2.5__*) Declares a leaf module.
This element takes the following attributes: +
* `organisation`: the organisation of the leaf module (defaults to *) +
* `module`: the name of the leaf module (defaults to *) +
* `revision`: the revision of the leaf module (defaults to *) +
* `branch`: the branch of the leaf module (default to *)
* `file`: a specific ivy.xml file to use as leaf module|0..n
|=======
== Parameters specified as nested elements
=== fileset
@ -132,3 +155,27 @@ Builds a list of `build.xml` files sorted according to the `ivy.xml` files found
----
Builds a list of `build.xml` files sorted according to the `ivy.xml` files found in an Ivy directory relative to those build files. Only `build.xml` files of modules which have dependencies (direct or transitive) on `mymodule` are put in the result list.
'''
[source,xml]
----
<ivy:buildlist reference="build-path" ivyfilepath="ivy/ivy.xml">
<root organisation="myorg" module="myapp" />
<fileset dir="projects" includes="**/build.xml"/>
</ivy:buildlist>
----
Builds a list of `build.xml` files sorted according to the `ivy.xml` files found in an Ivy directory relative to those build files. Only `build.xml` files of modules which are dependencies of `myorg#myapp` (either direct or transitive) are put in the result list.
'''
[source,xml]
----
<ivy:buildlist reference="build-path" ivyfilepath="ivy/ivy.xml">
<root file="/path/to/myapp-ivy.xml" />
<fileset dir="projects" includes="**/build.xml"/>
</ivy:buildlist>
----
Builds a list of `build.xml` files sorted according to the `ivy.xml` files found in an Ivy directory relative to those build files. Only `build.xml` files of modules which are dependencies defined in `/path/to/myapp-ivy.xml` (either direct or transitive) are put in the result list.

View File

@ -30,11 +30,14 @@ import java.util.Set;
import java.util.StringTokenizer;
import org.apache.ivy.Ivy;
import org.apache.ivy.core.IvyPatternHelper;
import org.apache.ivy.core.module.descriptor.DependencyDescriptor;
import org.apache.ivy.core.module.descriptor.ModuleDescriptor;
import org.apache.ivy.core.module.id.ModuleId;
import org.apache.ivy.core.settings.IvySettings;
import org.apache.ivy.core.sort.SortOptions;
import org.apache.ivy.plugins.matcher.MapMatcher;
import org.apache.ivy.plugins.matcher.PatternMatcher;
import org.apache.ivy.plugins.parser.ModuleDescriptorParserRegistry;
import org.apache.ivy.util.Message;
import org.apache.tools.ant.BuildException;
@ -62,6 +65,60 @@ public class IvyBuildList extends IvyTask {
}
}
public static final class BuildListModule {
private String organisation;
private String module;
private String revision;
private String branch;
private File file;
public String getOrganisation() {
return organisation;
}
public void setOrganisation(String organisation) {
this.organisation = organisation;
}
public String getModule() {
return module;
}
public void setModule(String module) {
this.module = module;
}
public String getRevision() {
return revision;
}
public void setRevision(String revision) {
this.revision = revision;
}
public String getBranch() {
return branch;
}
public void setBranch(String branch) {
this.branch = branch;
}
public File getFile() {
return file;
}
public void setFile(File file) {
this.file = file;
}
}
public static final String DESCRIPTOR_REQUIRED = "required";
private List<FileSet> buildFileSets = new ArrayList<>();
@ -78,10 +135,14 @@ public class IvyBuildList extends IvyTask {
private String root = "*";
private List<BuildListModule> roots = new ArrayList<>();
private boolean excludeRoot = false;
private String leaf = "*";
private List<BuildListModule> leafs = new ArrayList<>();
private String delimiter = ",";
private boolean excludeLeaf = false;
@ -110,6 +171,12 @@ public class IvyBuildList extends IvyTask {
this.root = root;
}
public BuildListModule createRoot() {
BuildListModule root = new BuildListModule();
roots.add(root);
return root;
}
public boolean isExcludeRoot() {
return excludeRoot;
}
@ -126,6 +193,12 @@ public class IvyBuildList extends IvyTask {
this.leaf = leaf;
}
public BuildListModule createLeaf() {
BuildListModule leaf = new BuildListModule();
leafs.add(leaf);
return leaf;
}
public boolean isExcludeLeaf() {
return excludeLeaf;
}
@ -172,28 +245,9 @@ public class IvyBuildList extends IvyTask {
List<File> noDescriptor = new ArrayList<>();
Collection<ModuleDescriptor> mds = new ArrayList<>();
Set<String> rootModuleNames = new LinkedHashSet<>();
if (!"*".equals(root)) {
StringTokenizer st = new StringTokenizer(root, delimiter);
while (st.hasMoreTokens()) {
rootModuleNames.add(st.nextToken());
}
}
Set<String> leafModuleNames = new LinkedHashSet<>();
if (!"*".equals(leaf)) {
StringTokenizer st = new StringTokenizer(leaf, delimiter);
while (st.hasMoreTokens()) {
leafModuleNames.add(st.nextToken());
}
}
Set<String> restartFromModuleNames = new LinkedHashSet<>();
if (!"*".equals(restartFrom)) {
StringTokenizer st = new StringTokenizer(restartFrom, delimiter);
// Only accept one (first) module
restartFromModuleNames.add(st.nextToken());
}
Set<MapMatcher> rootModules = convert(roots, root, settings);
Set<MapMatcher> leafModules = convert(leafs, leaf, settings);
Set<MapMatcher> restartFromModules = convert(Collections.<BuildListModule>emptyList(), restartFrom, settings);
for (FileSet fs : buildFileSets) {
DirectoryScanner ds = fs.getDirectoryScanner(getProject());
@ -225,19 +279,19 @@ public class IvyBuildList extends IvyTask {
}
}
List<ModuleDescriptor> leafModuleDescriptors = convertModuleNamesToModuleDescriptors(mds,
leafModuleNames, "leaf");
List<ModuleDescriptor> rootModuleDescriptors = convertModuleNamesToModuleDescriptors(mds,
rootModuleNames, "root");
List<ModuleDescriptor> restartFromModuleDescriptors = convertModuleNamesToModuleDescriptors(
mds, restartFromModuleNames, "restartFrom");
List<ModuleDescriptor> leafModuleDescriptors =
findModuleDescriptors(mds, leafModules, "leaf");
List<ModuleDescriptor> rootModuleDescriptors =
findModuleDescriptors(mds, rootModules, "root");
List<ModuleDescriptor> restartFromModuleDescriptors =
findModuleDescriptors(mds, restartFromModules, "restartFrom");
if (!rootModuleDescriptors.isEmpty()) {
Message.info("Filtering modules based on roots " + rootModuleNames);
Message.info("Filtering modules based on roots [" + extractModuleNames(rootModules) + "]");
mds = filterModulesFromRoot(mds, rootModuleDescriptors);
}
if (!leafModuleDescriptors.isEmpty()) {
Message.info("Filtering modules based on leafs " + leafModuleNames);
Message.info("Filtering modules based on leafs [" + extractModuleNames(leafModules) + "]");
mds = filterModulesFromLeaf(mds, leafModuleDescriptors);
}
@ -260,6 +314,7 @@ public class IvyBuildList extends IvyTask {
if (!restartFromModuleDescriptors.isEmpty()) {
boolean foundRestartFrom = false;
List<ModuleDescriptor> keptModules = new ArrayList<>();
// Only accept one (first) module
ModuleDescriptor restartFromModuleDescriptor = restartFromModuleDescriptors.get(0);
for (ModuleDescriptor md : sortedModules) {
if (md.equals(restartFromModuleDescriptor)) {
@ -289,6 +344,54 @@ public class IvyBuildList extends IvyTask {
getProject().setProperty("ivy.sorted.modules", order.toString());
}
private Set<MapMatcher> convert(List<BuildListModule> modulesList, String modulesString, IvySettings settings) {
Set<MapMatcher> result = new LinkedHashSet<>();
for (BuildListModule module : modulesList) {
File ivyFile = module.getFile();
if (ivyFile == null) {
String org = module.getOrganisation();
String name = module.getModule();
String rev = module.getRevision();
String branch = module.getBranch();
Map<String, String> attributes = new HashMap<>();
attributes.put(IvyPatternHelper.ORGANISATION_KEY, org == null ? PatternMatcher.ANY_EXPRESSION : org);
attributes.put(IvyPatternHelper.MODULE_KEY, name == null ? PatternMatcher.ANY_EXPRESSION : name);
attributes.put(IvyPatternHelper.MODULE_KEY, rev == null ? PatternMatcher.ANY_EXPRESSION : rev);
attributes.put(IvyPatternHelper.MODULE_KEY, branch == null ? PatternMatcher.ANY_EXPRESSION : branch);
result.add(new MapMatcher(attributes, settings.getMatcher(PatternMatcher.EXACT)));
} else {
try {
ModuleDescriptor md = ModuleDescriptorParserRegistry.getInstance()
.parseDescriptor(settings, ivyFile.toURI().toURL(),
doValidate(settings));
Map<String, String> attributes = new HashMap<>();
attributes.putAll(md.getModuleRevisionId().getAttributes());
attributes.put("resource", md.getResource().getName());
result.add(new MapMatcher(attributes, settings.getMatcher(PatternMatcher.EXACT)));
} catch (Exception e) {
throw new BuildException(e);
}
}
}
if (!"*".equals(modulesString)) {
StringTokenizer st = new StringTokenizer(modulesString, getDelimiter());
while (st.hasMoreTokens()) {
Map<String, String> attributes = new HashMap<>();
attributes.put(IvyPatternHelper.MODULE_KEY, st.nextToken());
result.add(new MapMatcher(attributes, settings.getMatcher(PatternMatcher.EXACT)));
}
}
return result;
}
private void onMissingDescriptor(File buildFile, File ivyFile, List<File> noDescriptor) {
switch (onMissingDescriptor) {
case OnMissingDescriptor.FAIL:
@ -312,38 +415,52 @@ public class IvyBuildList extends IvyTask {
}
}
private List<ModuleDescriptor> convertModuleNamesToModuleDescriptors(
Collection<ModuleDescriptor> mds, Set<String> moduleNames, String kind) {
private List<ModuleDescriptor> findModuleDescriptors(
Collection<ModuleDescriptor> mds, Set<MapMatcher> matchers, String kind) {
List<ModuleDescriptor> result = new ArrayList<>();
Set<String> foundModuleNames = new HashSet<>();
Set<MapMatcher> missingMatchers = new HashSet<>(matchers);
for (ModuleDescriptor md : mds) {
String name = md.getModuleRevisionId().getModuleId().getName();
if (moduleNames.contains(name)) {
foundModuleNames.add(name);
result.add(md);
Map<String, String> attributes = new HashMap<>();
attributes.putAll(md.getAttributes());
attributes.put("resource", md.getResource().getName());
for (MapMatcher matcher : matchers) {
if (matcher.matches(attributes)) {
missingMatchers.remove(matcher);
result.add(md);
}
}
}
if (foundModuleNames.size() < moduleNames.size()) {
Set<String> missingModules = new HashSet<>(moduleNames);
missingModules.removeAll(foundModuleNames);
StringBuilder missingNames = new StringBuilder();
String sep = "";
for (String name : missingModules) {
missingNames.append(sep);
missingNames.append(name);
sep = ", ";
}
if (!missingMatchers.isEmpty()) {
throw new BuildException("unable to find " + kind + " module(s) "
+ missingNames.toString() + " in build fileset");
+ extractModuleNames(missingMatchers) + " in build fileset");
}
return result;
}
private String extractModuleNames(Set<MapMatcher> matchers) {
StringBuilder result = new StringBuilder();
String sep = "";
for (MapMatcher matcher : matchers) {
result.append(sep);
Map<String, String> attributes = matcher.getAttributes();
String organisation = attributes.get(IvyPatternHelper.ORGANISATION_KEY);
if (organisation != null && !PatternMatcher.ANY_EXPRESSION.equals(organisation)) {
result.append(organisation);
result.append('#');
}
result.append(attributes.get(IvyPatternHelper.MODULE_KEY));
sep = ", ";
}
return result.toString();
}
/**
* Returns a collection of ModuleDescriptors that are contained in the input collection of
* ModuleDescriptors and upon which the root module depends