Add PlayTorch to Existing App

In this tutorial, you will learn how to add the PlayTorch core package to an existing React Native project.

If you have an existing React Native project and you want to add ML capabilities, you can add the react-native-pytorch-core package. This package includes all code needed to run ML inference, the Canvas, Camera, ImageUtil, and AudioUtil.

Installation

yarn add react-native-pytorch-core

On iOS you are done, but Android requires the following additional steps for the react-native-pytorch-core package to work.

Additional Assets for Metro

If the PyTorch Mobile models are part of the React Native bundle, the Metro configuration needs to be changed to resolve the ptl files.

note

This is only required if models are loaded from the bundle using require('./path/to/model.ptl'). It is not required if models are loaded from the mobile device's local file system or via a URL.

metro.config.js

// get defaults assetExts array
const defaultAssetExts = require('metro-config/src/defaults/defaults')
  .assetExts;

module.exports = {
  // ...

  resolver: {
    assetExts: [...defaultAssetExts, 'ptl'],
  },

  // ...
};

Additional steps on Android

For the react-native-pytorch-core React Native package to work on Android, it requires three changes to the gradle.properties and the two build.gradle files to increase JVM memory, add Sonatype repository, and packaging options with pick first rule.

Increase JVM Memory

Increase the memory for the JVM to avoid OutOfMemory exceptions during the packaging process.

./android/gradle.properties

org.gradle.jvmargs=-Xmx4g

Without the increased memory, the packaging process might fail with the following error:

* What went wrong:
Execution failed for task ':app:packageDebug'.
> A failure occurred while executing com.android.build.gradle.tasks.PackageAndroidArtifact$IncrementalSplitterRunnable
   > java.lang.OutOfMemoryError (no error message)

Update app build.gradle

Both React Native and PyTorch Mobile for Android use fbjni. For example, the versions for PlayTorch that are used for development are:

• React Native 0.64.3 uses fbjni 0.0.2
• PyTorch Mobile 1.12.2 uses fbjni 0.2.2.

So far, fbjni is forward compatible, which means it is ok to pick the latest version shipped with either of the two dependencies. At this point, it is fbjni 0.2.2. For Gradle to pick the right version, the android/app/build.gradle needs to have a few adjustments:

1. Add pickFirst rule to packagingOptions. This rule will pick the first shared object (dynamic) library. It will give higher priority to shared object libraries that are coming with direct app dependencies, which is why 2. is important.
2. Set up an extra directory for fbjni where the fbjni version 0.2.2 from the dependency added in 3. will be extracted. Also add the relevant task to the build.gradle file (see task extraJNILibs and tasks.whenTaskAdded after the dependencies definition)
3. Add fbjni 0.2.2 as direct app dependency.

See the build.gradle in the react-native-pytorch-core example app for a possible configuration.

info

The following error will show if pickFirst is not set:

* What went wrong:
Execution failed for task ':app:mergeDebugNativeLibs'.
> A failure occurred while executing com.android.build.gradle.internal.tasks.Workers$ActionFacade
   > More than one file was found with OS independent path 'lib/x86/libfbjni.so'

./android/app/build.gradle

android {
    // ...

    /**
     * Without the packaging options, it will result in the following build error:
     *
     * * What went wrong:
     * Execution failed for task ':app:mergeDebugNativeLibs'.
     * > A failure occurred while executing com.android.build.gradle.internal.tasks.Workers$ActionFacade
     *    > More than one file was found with OS independent path 'lib/x86/libfbjni.so'
     */
    packagingOptions {
        pickFirst '**/*.so'
    }
    sourceSets {
        main {
            jniLibs.srcDirs += ["$buildDir/extra-jniLibs/jni"]
        }
    }
    configurations {
        extraJNILibs
    }

    // ...
}

dependencies {
    // ...

    // Used to control the version of libfbjni.so packaged into the APK
    extraJNILibs("com.facebook.fbjni:fbjni:0.2.2")

    // ...
}

// ...

// Extract JNI shared libraries as project libraries. This assumes the target directory, $buildDir/extra-jniLibs, is added to the jniLibs.srcDirs configuration.
task extraJNILibs {
  doLast {
    configurations.extraJNILibs.files.each {
      def file = it.absoluteFile

      copy {
        from zipTree(file)
        into "$buildDir/extra-jniLibs" // temp location instead of "src/main/jniLibs"
        include "jni/**/*"
      }
    }
  }
}

tasks.whenTaskAdded { task ->
  if (task.name == 'mergeDebugJniLibFolders' || task.name == 'mergeReleaseJniLibFolders') {
    task.dependsOn(extraJNILibs)
  }
}

// ...

That should be all necessary Gradle build configuration changes!

Give us feedback

Share what we can improve!