Using Android NDK to Optimize Barcode Reading Performance

Previously, I shared an article demonstrating how to use Camera2 APIs and Dynamsoft Barcode Reader to build a simple Android barcode reader app. In that demo project, the barcode decoding part is implemented in Java, which apparently has room for improvement. If we can get the pointer to the native buffer of the camera frame, we can invoke native Barcode Reader APIs directly. This article shares how to write JNI code for Android barcode detection, as well as how to use Android NDK and CMake to build the C++ code.

Prerequisites

Android NDK and CMake

Open Android Studio and select Tools > SDK Manager > SDK Tools. Check NDK and CMake to install:

To learn NDK, you can visit https://github.com/googlesamples/android-ndk.

Dynamsoft Barcode Reader for Android

Download Dynamsoft Barcode Reader for Android.

Extract the DynamsoftBarcodeReaderAndroid.aar file from the package. To build a barcode reader app in Java, you just need to import the *.aar file as a module. Here I’m going to show you how to invoke native APIs, so open the aar file with a file archiver, such as 7-Zip. Extract a platform-compatible shared library from DynamsoftBarcodeReaderAndroid.aar\jni\<ARCH>\ libDynamsoftBarcodeReaderAndroid.so.

The Simple Barcode Reader Demo

Get the source code from GitHub:

git clone https://github.com/yushulx/android-camera2-barcode

Android Barcode Decoding Using JNI

Import the project into Android Studio. It is time to do the optimization.

Here are three native methods:

private native ArrayList<SimpleResult> readBarcode(long hBarcode, ByteBuffer byteBuffer, int width, int height, int stride);
private native long createBarcodeReader(String license);
private native void destroyBarcodeReader(long ndkBarcodeReader);

What I’m going to do is to instantiate the barcode reader object in C++ and save its memory address in Java.

Android Camera2 APIs provide an ImageReader class to acquire preview images from the camera. The returned data type is ByteBuffer rather than byte[]. A byte buffer is allocated from native code via JNI. When using the buffer for barcode detection in Java, we have to copy it into a byte array:

byte[] bytes = new byte[buffer.remaining()];
buffer.get(bytes);

In contrast, there’s no extra memory copy step in C++:

unsigned char * buffer = (unsigned char*)env->GetDirectBufferAddress(byteBuffer);

The underlying technology of Dynamsoft Barcode Reader for Android is also based on JNI. When calling the barcode decoding method in Java, it will first copy the Java byte array into a native buffer in order to invoke the C++ methods. Therefore, the optimization I can do is to reduce the image memory copy twice.

Create a src/main/cpp/android_main.cpp file:

#include <jni.h>
#include <cstring>
#include "DynamsoftBarcodeReader.h"
#include <android/log.h>

#define LOG_TAG "BarcodeReader"
#define LOGI(...) __android_log_print(ANDROID_LOG_INFO, LOG_TAG, __VA_ARGS__)
#define LOGE(...) __android_log_print(ANDROID_LOG_ERROR, LOG_TAG, __VA_ARGS__)
/**
 * BarcodeReader object:
 *
 */
void *pBarcodeReader = nullptr;

extern "C" JNIEXPORT jobject JNICALL
Java_com_example_android_camera2basic_Camera2BasicFragment_readBarcode(JNIEnv *env, jobject instance, jlong hBarcode,
                                                                       jobject byteBuffer, jint width, jint height, jint stride) {

    // ArrayList
    jclass classArray = env->FindClass("java/util/ArrayList");
    if (classArray == NULL) return NULL;
    jmethodID midArrayInit =  env->GetMethodID(classArray, "<init>", "()V");
    if (midArrayInit == NULL) return NULL;
    jobject objArr = env->NewObject(classArray, midArrayInit);
    if (objArr == NULL) return NULL;
    jmethodID midAdd = env->GetMethodID(classArray, "add", "(Ljava/lang/Object;)Z");
    if (midAdd == NULL) return NULL;

    // SimpleResult
    jclass cls = env->FindClass("com/example/android/camera2basic/Camera2BasicFragment$SimpleResult");
    if (NULL == cls) return NULL;
    jmethodID midInit = env->GetMethodID(cls, "<init>", "(Ljava/lang/String;Ljava/lang/String;)V");
    if (NULL == midInit) return NULL;


    unsigned char * buffer = (unsigned char*)env->GetDirectBufferAddress(byteBuffer);

    int ret = DBR_DecodeBuffer((void *)hBarcode, buffer, width, height, stride, IPF_NV21, "");

    if (ret) {
        LOGE("Detection error: %s", DBR_GetErrorString(ret));
//        return NULL;
    }

    STextResultArray *pResults = NULL;
    DBR_GetAllTextResults((void *)hBarcode, &pResults);
    if (pResults)
    {
        int count = pResults->nResultsCount;

        for (int i = 0; i < count; i++)
        {
            jobject newObj = env->NewObject(cls, midInit, env->NewStringUTF(pResults->ppResults[i]->pszBarcodeFormatString), env->NewStringUTF(pResults->ppResults[i]->pszBarcodeText));
            env->CallBooleanMethod(objArr, midAdd, newObj);
        }

        // release memory of barcode results
        DBR_FreeTextResults(&pResults);
    }
    return objArr;

}

extern "C" JNIEXPORT jlong JNICALL
Java_com_example_android_camera2basic_Camera2BasicFragment_createBarcodeReader(JNIEnv *env, jobject instance, jstring license) {
    if (!pBarcodeReader) {
        // Instantiate barcode reader object.
        pBarcodeReader = DBR_CreateInstance();

        // Initialize the license key.
        const char *nativeString = env->GetStringUTFChars(license, 0);
        DBR_InitLicense(pBarcodeReader, nativeString);
        env->ReleaseStringUTFChars(license, nativeString);
    }

    return (jlong)(pBarcodeReader);

}

extern "C" JNIEXPORT void JNICALL
Java_com_example_android_camera2basic_Camera2BasicFragment_destroyBarcodeReader(JNIEnv *env, jobject instance, jlong hBarcode) {

    if (hBarcode) {
        DBR_DestroyInstance((void *)hBarcode);
    }

}

Where is the DynamsoftBarcodeReader.h file? The header file does not exist in the SDK package for Android.  But don’t worry. Since Dynamsoft Barcode Reader is cross-platform, it is easy to get the header file from other editions.

We can compare the code change.

Reading barcodes from the Java layer:

ByteBuffer buffer = image.getPlanes()[0].getBuffer();
int nRowStride = image.getPlanes()[0].getRowStride();
int nPixelStride = image.getPlanes()[0].getPixelStride();
byte[] bytes = new byte[buffer.remaining()];
buffer.get(bytes);
TextResult[] results = mBarcodeReader.decodeBuffer(bytes, mImageReader.getWidth(), mImageReader.getHeight(), nRowStride * nPixelStride, EnumImagePixelFormat.IPF_NV21, "");

Reading barcodes from the JNI layer:

ByteBuffer buffer = image.getPlanes()[0].getBuffer();
int nRowStride = image.getPlanes()[0].getRowStride();
int nPixelStride = image.getPlanes()[0].getPixelStride();
ArrayList<SimpleResult> results = readBarcode(hBarcode, buffer, mImageReader.getWidth(), mImageReader.getHeight(), nRowStride * nPixelStride);

Here is the final look of the project structure:

Building C++ Code with Android NDK and CMake

Create a CMakeLists.txt file under src/main/cpp:

cmake_minimum_required(VERSION 3.4.1)

set(CMAKE_VERBOSE_MAKEFILE on)
set(CMAKE_ANDROID_ARCH_ABI armeabi-v7a)

link_directories("${CMAKE_CURRENT_SOURCE_DIR}")

add_library(dynamsoft_barcode SHARED
    ${CMAKE_CURRENT_SOURCE_DIR}/android_main.cpp)

# add include path
target_include_directories(dynamsoft_barcode PRIVATE ${COMMON_SOURCE_DIR})

# add lib dependencies
target_link_libraries(dynamsoft_barcode dl android log m DynamsoftBarcodeReaderAndroid)

Note: do not move link_directories down below add_library. If you do so, the build will fail:

Android NDK CMake Error:
error: cannot find -lDynamsoftBarcodeReaderAndroid

In the build.gradle file, add the following script:

defaultConfig {
    minSdkVersion 21
    targetSdkVersion 27
    ndk {
        abiFilters 'armeabi-v7a'
    }
    externalNativeBuild {
        cmake {
            arguments '-DANDROID_STL=c++_static', '-DANDROID_ABI=armeabi-v7a'
        }
    }
}

externalNativeBuild {
    cmake {
        version '3.10.2'
        path 'src/main/cpp/CMakeLists.txt'
    }
}

By default, the build will generate shared libraries including arm64-v8a, armeabi-v7a, x86 and x86_64. To only generate the matched share library, the ABI filter is required.

Now we can successfully build and run the barcode reader app:

Source Code

https://github.com/yushulx/android-camera2-barcode-ndk

The post Using Android NDK to Optimize Barcode Reading Performance appeared first on Code Pool.