-
Notifications
You must be signed in to change notification settings - Fork 3.8k
Description
Potentially a blocker for Arrow Integration into Calcite: CALCITE-2040, however it may be possible to move forward without M1 Mac support.
potentially somewhat related to ARROW-11135
Getting an instance of the JNILoader throw a Unsatisfied Link Error when it tries to load the libgandiva_jni.dylib that it has extracted from the jar into a temporary directory.
Simplified error:
Exception in thread "main" java.lang.UnsatisfiedLinkError: /tmp_dir/libgandiva_jni.dylib_uuid: dlopen(/tmp_dir/libgandiva_jni.dylib_uuid, 0x0001): tried: '/tmp_dir/libgandiva_jni.dylib_uuid' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e'))
Full error and stack trace:
Exception in thread "main" java.lang.UnsatisfiedLinkError: /private/var/folders/fj/63_6n5dx10n4b5x7jtdj6tvh0000gn/T/libgandiva_jni.dylib526a47e1-7306-440f-8bbf-378877abe5fe: dlopen(/private/var/folders/fj/63_6n5dx10n4b5x7jtdj6tvh0000gn/T/libgandiva_jni.dylib526a47e1-7306-440f-8bbf-378877abe5fe, 0x0001): tried: '/private/var/folders/fj/63_6n5dx10n4b5x7jtdj6tvh0000gn/T/libgandiva_jni.dylib526a47e1-7306-440f-8bbf-378877abe5fe' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e'))
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1950)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1832)
at java.lang.Runtime.load0(Runtime.java:811)
at java.lang.System.load(System.java:1088)
at org.apache.arrow.gandiva.evaluator.JniLoader.loadGandivaLibraryFromJar(JniLoader.java:74)
at org.apache.arrow.gandiva.evaluator.JniLoader.setupInstance(JniLoader.java:63)
at org.apache.arrow.gandiva.evaluator.JniLoader.getInstance(JniLoader.java:53)
at org.apache.arrow.gandiva.evaluator.JniLoader.getDefaultConfiguration(JniLoader.java:144)
at org.apache.arrow.gandiva.evaluator.Filter.make(Filter.java:67)
at io.acme.Main.main(Main.java:26)
This example loads three libraries from mavencentral using gradle:
repositories {
mavenCentral()
}
dependencies {
implementation("org.apache.arrow:arrow-memory-netty:8.0.0")
implementation("org.apache.arrow:arrow-vector:8.0.0")
implementation("org.apache.arrow.gandiva:arrow-gandiva:8.0.0")
}
Example code:
public class Main {
public static void main(String[] args) throws GandivaException {
Field field = new Field("int_field", FieldType.nullable(new ArrowType.Int(32, true)), null);
Schema schema = makeSchema(field);
Condition condition = makeCondition(field);
Filter.make(schema, condition);
}
private static Schema makeSchema(Field field) {
List<Field> fieldList = new ArrayList<>();
fieldList.add(field);
return new Schema(fieldList, null);
}
private static Condition makeCondition(Field f) {
List<TreeNode> treeNodes = new ArrayList<>(2);
treeNodes.add(TreeBuilder.makeField(f));
treeNodes.add(TreeBuilder.makeLiteral(4));
TreeNode comparison = TreeBuilder.makeFunction("less_than", treeNodes, new ArrowType.Bool());
return TreeBuilder.makeCondition(comparison);
}
}
While I haven't tested this exact example, a similar example executes without issue on an intel x86 mac.
Reporter: Jonathan Swenson
Assignee: Rok Mihevc / @rok
Watchers: Rok Mihevc / @rok
Related issues:
- Failure loading arrow_dataset_jni library on Mac M1 (duplicates)
- Create adapter for Apache Arrow (is depended upon by)
Note: This issue was originally created as ARROW-16608. Please see the migration documentation for further details.