Skip to content

[Gandiva][Java] Unsatisfied Link Error on M1 Mac when using mavencentral artifacts #31962

@asfimport

Description

@asfimport

Potentially a blocker for Arrow Integration into Calcite: CALCITE-2040, however it may be possible to move forward without M1 Mac support. 

potentially somewhat related to ARROW-11135

Getting an instance of the JNILoader throw a Unsatisfied Link Error when it tries to load the libgandiva_jni.dylib that it has extracted from the jar into a temporary directory. 

Simplified error:

Exception in thread "main" java.lang.UnsatisfiedLinkError: /tmp_dir/libgandiva_jni.dylib_uuid: dlopen(/tmp_dir/libgandiva_jni.dylib_uuid, 0x0001): tried: '/tmp_dir/libgandiva_jni.dylib_uuid' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e'))

 

Full error and stack trace:

Exception in thread "main" java.lang.UnsatisfiedLinkError: /private/var/folders/fj/63_6n5dx10n4b5x7jtdj6tvh0000gn/T/libgandiva_jni.dylib526a47e1-7306-440f-8bbf-378877abe5fe: dlopen(/private/var/folders/fj/63_6n5dx10n4b5x7jtdj6tvh0000gn/T/libgandiva_jni.dylib526a47e1-7306-440f-8bbf-378877abe5fe, 0x0001): tried: '/private/var/folders/fj/63_6n5dx10n4b5x7jtdj6tvh0000gn/T/libgandiva_jni.dylib526a47e1-7306-440f-8bbf-378877abe5fe' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e'))
    at java.lang.ClassLoader$NativeLibrary.load(Native Method)
    at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1950)
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1832)
    at java.lang.Runtime.load0(Runtime.java:811)
    at java.lang.System.load(System.java:1088)
    at org.apache.arrow.gandiva.evaluator.JniLoader.loadGandivaLibraryFromJar(JniLoader.java:74)
    at org.apache.arrow.gandiva.evaluator.JniLoader.setupInstance(JniLoader.java:63)
    at org.apache.arrow.gandiva.evaluator.JniLoader.getInstance(JniLoader.java:53)
    at org.apache.arrow.gandiva.evaluator.JniLoader.getDefaultConfiguration(JniLoader.java:144)
    at org.apache.arrow.gandiva.evaluator.Filter.make(Filter.java:67)
    at io.acme.Main.main(Main.java:26) 

 

This example loads three libraries from mavencentral using gradle: 

repositories {
    mavenCentral()
}

dependencies {
    implementation("org.apache.arrow:arrow-memory-netty:8.0.0")
    implementation("org.apache.arrow:arrow-vector:8.0.0")
    implementation("org.apache.arrow.gandiva:arrow-gandiva:8.0.0")
} 

Example code: 

public class Main {
  public static void main(String[] args) throws GandivaException {
    Field field = new Field("int_field", FieldType.nullable(new ArrowType.Int(32, true)), null);

    Schema schema = makeSchema(field);
    Condition condition = makeCondition(field);

    Filter.make(schema, condition);
  }

  private static Schema makeSchema(Field field) {
    List<Field> fieldList = new ArrayList<>();
    fieldList.add(field);

    return new Schema(fieldList, null);
  }

  private static Condition makeCondition(Field f) {
    List<TreeNode> treeNodes = new ArrayList<>(2);
    treeNodes.add(TreeBuilder.makeField(f));
    treeNodes.add(TreeBuilder.makeLiteral(4));
    TreeNode comparison = TreeBuilder.makeFunction("less_than", treeNodes, new ArrowType.Bool());
    return TreeBuilder.makeCondition(comparison);
  }
} 

While I haven't tested this exact example, a similar example executes without issue on an intel x86 mac.

Reporter: Jonathan Swenson
Assignee: Rok Mihevc / @rok
Watchers: Rok Mihevc / @rok

Related issues:

Note: This issue was originally created as ARROW-16608. Please see the migration documentation for further details.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions