Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace lib:/// scheme by mvn:/// and file:/// and jar+file:/// everywhere #1969

Open
wants to merge 47 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
8ff9eae
added mavenize method to shorten jar paths in the local maven reposit…
jurgenvinju Jun 10, 2024
0608349
made mavenize and jarify available to Rascal programmers
jurgenvinju Jun 10, 2024
5a5d5a4
cleanup of PathConfig and added diagnostics to the dependency resolut…
jurgenvinju Jun 11, 2024
086aa60
not finished yet but getting there with the PathConfig refurbishing
jurgenvinju Jun 11, 2024
37e3c96
implemented Messages
jurgenvinju Jun 12, 2024
002e2f0
Merge branch 'main' into replace-lib-by-mvn-and-others
jurgenvinju Jun 17, 2024
f0b8709
Merge branch 'main' into replace-lib-by-mvn-and-others
jurgenvinju Jun 17, 2024
4d8ba06
cleaning up after changing PathConfig. Also removed unused imports he…
jurgenvinju Jun 17, 2024
b212c7e
removed the lib:/// scheme from the implementation
jurgenvinju Jun 17, 2024
f879803
added target folder back to the classpath for loading Java classes
jurgenvinju Jun 17, 2024
339eaea
Merge branch 'main' into replace-lib-by-mvn-and-others
jurgenvinju Jun 24, 2024
3034ebd
the shell evaluator factory and junit evaluator creator now print the…
jurgenvinju Jun 24, 2024
e3248de
some cleanup
jurgenvinju Jun 24, 2024
1ed3643
improved message printing
jurgenvinju Jun 27, 2024
b83b6f3
improved message printing
jurgenvinju Jun 27, 2024
66fd82b
improved configuration checking code
jurgenvinju Jun 27, 2024
107f687
improved error messages
jurgenvinju Jun 27, 2024
63ea033
finetuning interpreter mode wrt std library linkage
jurgenvinju Jun 28, 2024
f767142
changed warning message
jurgenvinju Jun 28, 2024
38bdf96
The std scheme is not opaque anymore but transparant
jurgenvinju Jun 28, 2024
a14a82a
added more comments
jurgenvinju Jun 28, 2024
72b2ba6
Merge branch 'main' into replace-lib-by-mvn-and-others
jurgenvinju Sep 14, 2024
d076cf0
fixed false positive project name error if the project:// scheme is used
jurgenvinju Sep 14, 2024
5ea2006
removed unused import
jurgenvinju Sep 16, 2024
ac8080f
fixed issue in std:// resolver in the case where we are applying it w…
jurgenvinju Sep 16, 2024
11739c4
added proper warning for conflicting rascal-lsp instances on the clas…
jurgenvinju Sep 16, 2024
cb4b38e
fixed typo
jurgenvinju Sep 16, 2024
4597800
removed legacy usages of javaCompilerPath
jurgenvinju Sep 16, 2024
561ce9e
removed remnants of Require-Libraries
jurgenvinju Sep 24, 2024
e9fdfe0
check for left-over Require-Libraries warning fixed
jurgenvinju Sep 24, 2024
4221cb0
optimization for the case where the m2 folder is not in the home dire…
jurgenvinju Sep 24, 2024
88b88fb
exposed new classpath resolution functions from PathConfig in util::R…
jurgenvinju Sep 24, 2024
27f91c0
turned off tutor temporarily
jurgenvinju Sep 24, 2024
b437fbf
necessary tools for rascal-lsp
jurgenvinju Sep 24, 2024
1596860
retrieved more utility functions required in rascal-lsp for some reason
jurgenvinju Sep 24, 2024
e4aa0d0
used constants better
jurgenvinju Sep 24, 2024
30dc484
rename
jurgenvinju Sep 24, 2024
968d013
renamed invalid to unknown location and removed the unused URI version
jurgenvinju Sep 24, 2024
2621744
resolved the comments by @davylandman
jurgenvinju Sep 24, 2024
564dde9
mvn hostname syntax changed from ! separators to ~ for compatibility …
jurgenvinju Sep 24, 2024
3fd5e12
replaced ~ separator by -- for compliance with also the DNS standards
jurgenvinju Sep 24, 2024
18581ea
comment fixed
jurgenvinju Sep 24, 2024
3a16fd9
fix minor hickups after changing mvn scheme syntax and confusions wit…
jurgenvinju Sep 25, 2024
497f6ca
rascalMF generation was broken (no empty line at end). Also factored …
jurgenvinju Sep 25, 2024
d571153
refactoring
jurgenvinju Sep 25, 2024
4b6dcb9
removed duplicate addition of rascal runtime in case of a dependency…
jurgenvinju Sep 25, 2024
6637721
added two print functions to PathConfig for reuse purposes
jurgenvinju Sep 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@
<goal>package</goal>
</goals>
</execution>
<execution>
<!--<execution>
<id>default-cli</id>
<phase>compile</phase>
<goals>
Expand All @@ -189,7 +189,7 @@
<ignore>${project.basedir}/src/org/rascalmpl/library/lang/rascal</ignore>
</ignores>
</configuration>
</execution>
</execution> -->
</executions>
</plugin>
<plugin>
Expand Down
1 change: 1 addition & 0 deletions src/org/rascalmpl/interpreter/Evaluator.java
Original file line number Diff line number Diff line change
Expand Up @@ -1174,6 +1174,7 @@ public Result<IValue> eval(IRascalMonitor monitor, Declaration declaration) {
* @param string
*/
public void doImport(IRascalMonitor monitor, String... string) {
assert monitor != null;
IRascalMonitor old = setMonitor(monitor);
interrupt = false;
try {
Expand Down
86 changes: 28 additions & 58 deletions src/org/rascalmpl/interpreter/utils/RascalManifest.java
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.Enumeration;
Expand All @@ -18,7 +17,6 @@
import java.util.jar.Manifest;
import java.util.zip.ZipEntry;

import org.rascalmpl.library.util.PathConfig;
import org.rascalmpl.uri.URIResolverRegistry;
import org.rascalmpl.uri.URIUtil;
import org.rascalmpl.uri.jar.JarURIResolver;
Expand All @@ -41,12 +39,11 @@ public class RascalManifest {
protected static final String SOURCE = "Source";
protected static final String META_INF = "META-INF";
public static final String META_INF_RASCAL_MF = META_INF + "/RASCAL.MF";
public static final String META_INF_MANIFEST_MF = META_INF + "/MANIFEST.MF";
protected static final String MAIN_MODULE = "Main-Module";
protected static final String MAIN_FUNCTION = "Main-Function";
protected static final String PROJECT_NAME = "Project-Name";
protected static final String REQUIRE_BUNDLES = "Require-Bundles";
protected static final String REQUIRE_LIBRARIES = "Require-Libraries";


public static String getRascalVersionNumber() {
try {
Enumeration<URL> resources = RascalManifest.class.getClassLoader().getResources("META-INF/MANIFEST.MF");
Expand All @@ -61,11 +58,28 @@ public static String getRascalVersionNumber() {
}
}

return "Rascal version not specified in META-INF/MANIFEST.MF???";
return "Not specified";
} catch (IOException e) {
return "unknown (due to " + e.getMessage();
}
}

/**
* This looks into the META-INF/MANIFEST.MF file for a Name and Specification-Version
*/
public String getManifestVersionNumber(ISourceLocation project) throws IOException {
Manifest mf = new Manifest(javaManifest(project));

String bundleName = mf.getMainAttributes().getValue("Name");
if (bundleName != null && bundleName.equals("rascal")) {
String result = mf.getMainAttributes().getValue("Specification-Version");
if (result != null) {
return result;
}
}

return "unknown";
}

public Manifest getDefaultManifest(String projectName) {
Manifest manifest = new Manifest();
Expand All @@ -75,7 +89,6 @@ public Manifest getDefaultManifest(String projectName) {
mainAttributes.put(new Attributes.Name(MAIN_MODULE), DEFAULT_MAIN_MODULE);
mainAttributes.put(new Attributes.Name(MAIN_FUNCTION), DEFAULT_MAIN_FUNCTION);
mainAttributes.put(new Attributes.Name(PROJECT_NAME), projectName);
mainAttributes.put(new Attributes.Name(REQUIRE_LIBRARIES), "");
return manifest;
}

Expand Down Expand Up @@ -152,13 +165,6 @@ public String getProjectName(File jarFile) {
return getManifestProjectName(manifest(jarFile));
}

/**
* @return a list of bundle names this jar depends on, or 'null' if none is configured.
*/
public List<String> getRequiredLibraries(JarInputStream jarStream) {
return getManifestRequiredLibraries(manifest(jarStream));
}

/**
* @return the name of the main module of a deployment unit, or 'null' if none is configured.
*/
Expand Down Expand Up @@ -195,20 +201,6 @@ public boolean hasMainModule(Class<?> clazz) {
return getManifestMainModule(manifest(clazz)) != null;
}

/**
* @return a list of bundle names this jar depends on, or 'null' if none is configured.
*/
public List<String> getRequiredLibraries(File jarFile) {
return getManifestRequiredLibraries(manifest(jarFile));
}

/**
* @return a list of bundle names this jar depends on, or 'null' if none is configured.
*/
public List<String> getRequiredLibraries(Class<?> clazz) {
return getManifestRequiredLibraries(manifest(clazz));
}

/**
* @return a list of paths relative to the root of the jar, if no such option is configured
* it will return ["src"].
Expand Down Expand Up @@ -238,17 +230,6 @@ public String getManifestMainModule(InputStream project) {
public String getManifestMainFunction(InputStream project) {
return getManifestAttribute(project, MAIN_FUNCTION, null);
}

/**
* @return a list of bundle names this jar depends on, or 'null' if none is configured.
*/
public List<String> getManifestRequiredLibraries(InputStream project) {
return getManifestAttributeList(project, REQUIRE_LIBRARIES, null);
}

public List<String> getManifestRequiredLibraries(ISourceLocation root) {
return getManifestAttributeList(manifest(root), REQUIRE_LIBRARIES, null);
}

public InputStream manifest(Class<?> clazz) {
return clazz.getResourceAsStream("/" + META_INF_RASCAL_MF);
Expand All @@ -262,6 +243,14 @@ public InputStream manifest(ISourceLocation root) {
}
}

public InputStream javaManifest(ISourceLocation root) {
try {
return URIResolverRegistry.getInstance().getInputStream(URIUtil.getChildLocation(JarURIResolver.jarify(root), META_INF_MANIFEST_MF));
} catch (IOException e) {
return null;
}
}

public String getProjectName(ISourceLocation root) {
return getManifestProjectName(manifest(root));
}
Expand All @@ -270,25 +259,6 @@ public List<String> getSourceRoots(ISourceLocation root) {
return getManifestSourceRoots(manifest(root));
}

public List<String> getRequiredLibraries(ISourceLocation root) {
return getManifestRequiredLibraries(manifest(root));
}

public PathConfig makePathConfig(ISourceLocation root) throws IOException {
List<ISourceLocation> libs = new ArrayList<>();
List<ISourceLocation> srcs = new ArrayList<>();

ISourceLocation binFolder = URIUtil.getChildLocation(root, "bin");
libs.add(binFolder);

RascalManifest mf = new RascalManifest();
for (String src : mf.getSourceRoots(root)) {
srcs.add(URIUtil.getChildLocation(root, src));
}

return new PathConfig(srcs, libs, binFolder);
}

public InputStream manifest(JarInputStream stream) {
JarEntry next = null;
ByteArrayOutputStream out = new ByteArrayOutputStream();
Expand Down
8 changes: 8 additions & 0 deletions src/org/rascalmpl/library/Location.rsc
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,14 @@ loc relativize(list[loc] haystack, loc needle) {
}
}

@synopsis{Shortens an absolute path to a jar inside the local maven repository.}
@javaClass{org.rascalmpl.library.Prelude}
java loc mavenize(loc jar);

@synopsis{If the location points to a jar file, then this modifies the scheme and the path to point _inside_ of the jar.}
@javaClass{org.rascalmpl.library.Prelude}
java loc jarify(loc jar);

@synopsis{Convert Windows path syntax to a `loc` value}
@description{
This conversion supports generic Windows path syntax, including:
Expand Down
143 changes: 143 additions & 0 deletions src/org/rascalmpl/library/Messages.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
package org.rascalmpl.library;

import java.io.PrintWriter;
import java.util.stream.Collectors;
import java.util.stream.Stream;

import org.rascalmpl.values.IRascalValueFactory;

import io.usethesource.vallang.IConstructor;
import io.usethesource.vallang.IList;
import io.usethesource.vallang.ISourceLocation;
import io.usethesource.vallang.IString;
import io.usethesource.vallang.IValue;
import io.usethesource.vallang.IValueFactory;
import io.usethesource.vallang.type.TypeFactory;
import io.usethesource.vallang.type.TypeStore;


/**
* Java API for the messages in the standard library module `Message`
*
* This is the standard format for all error messages in Rascal projects and beyond.
* Since some low-level core code also produces messages that should end up in UI,
* we write here a bridge between the Java and Rascal representation.
*
* TODO Later when the standard library is bootstrapped, this code might be replaced
* by the generated code from the compiler for the `Message` module.
*/
public class Messages {
private static final TypeFactory tf = TypeFactory.getInstance();
private static final IValueFactory vf = IRascalValueFactory.getInstance();
private static final TypeStore ts = new TypeStore();

// These declarations mirror the data definition in the `Message` root module of the standard library.
private static final io.usethesource.vallang.type.Type Message = tf.abstractDataType(ts, "Message");
private static final io.usethesource.vallang.type.Type Message_info = tf.constructor(ts, Message, "info", tf.stringType(), "msg", tf.sourceLocationType(), "at");
private static final io.usethesource.vallang.type.Type Message_warning = tf.constructor(ts, Message, "warning", tf.stringType(), "msg", tf.sourceLocationType(), "at");
private static final io.usethesource.vallang.type.Type Message_error = tf.constructor(ts, Message, "error", tf.stringType(), "msg", tf.sourceLocationType(), "at");

public static IValue info(String message, ISourceLocation loc) {
return vf.constructor(Message_info, vf.string(message), loc);
}

public static IValue warning(String message, ISourceLocation loc) {
return vf.constructor(Message_warning, vf.string(message), loc);
}

public static IValue error(String message, ISourceLocation loc) {
return vf.constructor(Message_error, vf.string(message), loc);
}

public static void write(IList messages, PrintWriter out) {
int maxLine = 0;
int maxColumn = 0;

for (IValue error : messages) {
ISourceLocation loc = (ISourceLocation) ((IConstructor) error).get("at");
if (loc.hasLineColumn()) {
maxLine = Math.max(loc.getBeginLine(), maxLine);
maxColumn = Math.max(loc.getBeginColumn(), maxColumn);
}
}

int lineWidth = (int) Math.log10(maxLine + 1) + 1;
int colWidth = (int) Math.log10(maxColumn + 1) + 1;

Stream<IConstructor> sortedStream = messages.stream()
.map(IConstructor.class::cast)
.sorted((m1, m2) -> {
ISourceLocation l1 = (ISourceLocation) m1.get("at");
ISourceLocation l2 = (ISourceLocation) m2.get("at");

if (!l1.getScheme().equals(l2.getScheme())) {
jurgenvinju marked this conversation as resolved.
Show resolved Hide resolved
return l1.getScheme().compareTo(l2.getScheme());
}

if (!l1.getAuthority().equals(l2.getAuthority())) {
return l1.getAuthority().compareTo(l2.getAuthority());
}

if (!l1.getPath().equals(l2.getPath())) {
return l1.getPath().compareTo(l2.getPath());
}

if (l1.hasLineColumn() && l2.hasLineColumn()) {
if (l1.getBeginLine() == l2.getBeginLine()) {
return Integer.compare(l1.getBeginColumn(), l2.getBeginColumn());
}
else {
return Integer.compare(l1.getBeginLine(), l2.getBeginLine());
}
}
else if (l1.hasOffsetLength() && l2.hasOffsetLength()) {
return Integer.compare(l1.getOffset(), l2.getOffset());
}
else if (l1.hasOffsetLength()) {
return -1;
}
else if (l2.hasOffsetLength()) {
return 1;
}

return 0;
});

for (IConstructor msg : sortedStream.collect(Collectors.toList())) {
String type = msg.getName();
boolean isError = type.equals("error");
boolean isWarning = type.equals("warning");

ISourceLocation loc = (ISourceLocation) msg.get("at");
int col = 0;
int line = 0;
if (loc.hasLineColumn()) {
col = loc.getBeginColumn();
line = loc.getBeginLine();
}

String output
= loc.getPath()
+ ":"
+ String.format("%0" + lineWidth + "d", line)
+ ":"
+ String.format("%0" + colWidth + "d", col)
+ ": "
+ ((IString) msg.get("msg")).getValue()
;

if (isError) {
out.println("[ERROR] " + output);
}
else if (isWarning) {
out.println("[WARNING] " + output);
}
else {
out.println("[INFO] " + output);
}
}

out.flush();
return;
}
}
10 changes: 10 additions & 0 deletions src/org/rascalmpl/library/Prelude.java
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,8 @@
import org.rascalmpl.uri.ISourceLocationWatcher.ISourceLocationChangeType;
import org.rascalmpl.uri.ISourceLocationWatcher.ISourceLocationChanged;
import org.rascalmpl.uri.ISourceLocationWatcher.ISourceLocationType;
import org.rascalmpl.uri.file.MavenRepositoryURIResolver;
import org.rascalmpl.uri.jar.JarURIResolver;
import org.rascalmpl.uri.LogicalMapResolver;
import org.rascalmpl.uri.URIResolverRegistry;
import org.rascalmpl.uri.URIUtil;
Expand Down Expand Up @@ -3673,6 +3675,14 @@ public ISourceLocation resolveLocation(ISourceLocation loc) {
}
}

public ISourceLocation mavenize(ISourceLocation jar) {
return MavenRepositoryURIResolver.mavenize(jar);
}

public ISourceLocation jarify(ISourceLocation jar) {
return JarURIResolver.jarify(jar);
}

public ISet findResources(IString fileName) {
return resourceProvider.findResources(fileName.getValue()).stream().collect(values.setWriter());
}
Expand Down
2 changes: 1 addition & 1 deletion src/org/rascalmpl/library/lang/csv/IO.java
Original file line number Diff line number Diff line change
Expand Up @@ -387,7 +387,7 @@ public IValue visitSet(Type type) throws RuntimeException {
}
@Override
public IValue visitSourceLocation(Type type) throws RuntimeException {
return values.sourceLocation(URIUtil.invalidURI());
return URIUtil.unknownLocation();
}
@Override
public IValue visitString(Type type) throws RuntimeException {
Expand Down
2 changes: 1 addition & 1 deletion src/org/rascalmpl/library/lang/java/m3/AST.rsc
Original file line number Diff line number Diff line change
Expand Up @@ -414,7 +414,7 @@ public set[Declaration] createAstsFromMavenProject(loc project, bool collectBind
throw "<project> is not a valid directory";
}

classPaths = getProjectPathConfig(project).javaCompilerPath;
classPaths = getProjectPathConfig(project).libs;
sourcePaths = getPaths(project, "java");
return createAstsFromFiles({ p | sp <- sourcePaths, p <- find(sp, "java"), isFile(p)}, collectBindings, sourcePath = [*findRoots(sourcePaths)], classPath = classPaths, errorRecovery = errorRecovery, javaVersion = javaVersion);
}
2 changes: 1 addition & 1 deletion src/org/rascalmpl/library/lang/java/m3/Core.rsc
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,7 @@ M3 createM3FromMavenProject(loc project, bool errorRecovery = false, bool includ
throw "<project> is not a valid directory";
}

list[loc] classPaths = getProjectPathConfig(project).javaCompilerPath;
list[loc] classPaths = getProjectPathConfig(project).libs;

sourcePaths = getPaths(project, "java");
M3 result = composeJavaM3(project, createM3sFromFiles({p | sp <- sourcePaths, p <- find(sp, "java"), isFile(p)}, errorRecovery = errorRecovery, sourcePath = [*findRoots(sourcePaths)], classPath = classPaths, javaVersion = javaVersion));
Expand Down
Loading
Loading