Generally, while doing barcode detection, we pass color images encoded in RGB or YUV to some barcode decoding APIs, in which the image data will be converted from color to grayscale and then from grayscale to binary. Operating pixels takes much CPU time. To accelerate image processing, we can utilize GPU. Dynamsoft JavaScript barcode SDK is powerful for developing web barcode scanning apps, but it is designed for CPU only so far. In this article, I will show you how to use WebGL to convert color image to grayscale image in order to reduce the total CPU time cost when reading barcodes from camera video stream in web apps.
Building Web Barcode Scanning Apps in 5 Minutes
Dynamsoft JavaScript Barcode SDK is available on npmjs.com:
https://www.npmjs.com/package/dynamsoft-javascript-barcode
The “hello world” sample is pretty simple:
<!DOCTYPE html>
<html>
<body>
<script src="https://cdn.jsdelivr.net/npm/dynamsoft-javascript-barcode@7.4.0-v1/dist/dbr.js" data-productKeys="PRODUCT-KEYS"></script>
<script>
let scanner = null;
(async()=>{
scanner = await Dynamsoft.BarcodeScanner.createInstance();
scanner.onFrameRead = results => {console.log(results);};
scanner.onUnduplicatedRead = (txt, result) => {alert(txt);};
await scanner.show();
})();
</script>
</body>
</html>
As you run the code, you can see the camera view and check the barcode reading results in the developer console. The BarcodeScanner class has done everything for you.
Alternatively, you can manually read the image data from the video element and programmatically call the decodeBuffer() method with the BarcodeReader class:
var barcodereader = null;
(async()=>{
barcodereader = await Dynamsoft.BarcodeReader.createInstance();
await barcodereader.updateRuntimeSettings('speed');
let settings = await barcodereader.getRuntimeSettings();
settings.deblurLevel = 0;
barcodereader.updateRuntimeSettings(settings);
})();
let canvas2d = document.createElement('canvas');
canvas2d.width = width;
canvas2d.height = height;
var ctx2d = canvas2d.getContext('2d');
ctx2d.drawImage(videoElement, 0, 0, width, height);
buffer = ctx2d.getImageData(0, 0, width, height).data;
if (barcodereader){
barcodereader
.decodeBuffer(
buffer,
width,
height,
width * 4,
Dynamsoft.EnumImagePixelFormat.IPF_ARGB_8888
)
.then((results) => {
showResults(results);
});
}
Converting Video Color Frame to Grayscale Image with WebGL
Theoretically, the processing speed of grayscale images is faster than that of color images in the same size, because a color image has 4 channels, whereas a grayscale image has only 1 channel. If the buffer is a grayscale image, the above code can be changed as follows:
barcodereader
.decodeBuffer(
gray,
width,
height,
width,
Dynamsoft.EnumImagePixelFormat.IPF_GrayScaled
)
.then((results) => {
showResults(results);
});
The next question is how I can get the grayscale image buffer with WebGL? I found a website named WebGLFundamentals is handy for learning WebGL.
First, I created a shader program to convert color to grayscale:
<!-- https://gist.github.com/Volcanoscar/4a9500d240497d3c0228f663593d167a -->
<script id="drawImage-fragment-shader" type="x-shader/x-fragment">
precision mediump float;
varying vec2 v_texcoord;
uniform sampler2D u_texture;
uniform float u_colorFactor;
void main() {
vec4 sample = texture2D(u_texture, v_texcoord);
float grey = 0.21 * sample.r + 0.71 * sample.g + 0.07 * sample.b;
gl_FragColor = vec4(sample.r * u_colorFactor + grey * (1.0 - u_colorFactor), sample.g * u_colorFactor + grey * (1.0 - u_colorFactor), sample.b * u_colorFactor + grey * (1.0 - u_colorFactor), 1.0);
}
</script>
Next, I used the draw() function provided by the online tutorial to bind video element to a WebGL texture:
var drawInfo = {
x: 0,
y: 0,
dx: 1,
dy: 1,
textureInfo: loadImageAndCreateTextureInfo(videoElement)
};
draw(drawInfo);
Then I read the image data to a Uint8Array:
buffer = new Uint8Array(width * height * 4);
gl.readPixels(
0,
0,
gl.drawingBufferWidth,
gl.drawingBufferHeight,
gl.RGBA,
gl.UNSIGNED_BYTE,
buffer
);
An interesting thing is the output image is upside down. To flip the image data along its vertical axis, you can call gl.pixelStorei():
function drawImage(tex, texWidth, texHeight, dstX, dstY) {
gl.bindTexture(gl.TEXTURE_2D, tex);
gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, true);
Finally, extract the grayscale data from the buffer:
gray = new Uint8Array(width * height);
let gray_index = 0;
for (i = 0; i < width * height * 4; i += 4) {
gray[gray_index++] = buffer[i];
}
Here is the screenshot of running the app in my web browser.
Barcode Decoding Performance Comparison: Grayscale Image vs Color Image
To measure the performance difference, we can use the formula:
total time = image data obtaining time + barcode decoding time
I made a simple test for 640×480 video stream.
As you can see, the time cost of getting image data from WebGL (grayscale image) is approximately 5x slower than that from canvas (color image) on my PC, for GPU data transfer takes time. However, the total CPU time cost of using WebGL for barcode decoding is less.
Source Code
https://github.com/yushulx/WebGL-JavaScript-Barcode
The post How to Use WebGL to Accelerate Web Barcode Decoding Speed appeared first on Dynamsoft Developers.