hugh 的个人博客

NodeJs如何解析avro二进制数据

背景: 做监控系统时,需要接受kafaka中传输的监控数据,传输来源采用avro编码 + application/octet-stream 二进制传输

处理方法:

1. 选择合适的框架 avscapi文档

npm install avsc


2. 具体处理

// 定义长度标识
var i =0;
if("application/octet-stream" == req.headers['content-type']) {
            i = req.body.length
        }

// 定义schema
const type = avro.Type.forSchema({
"namespace": "xxx",
"name": "Model",
"type": "record",
"fields": [
{
"name": "appId",
"type": "string"
}...]

// 分段解析数据, 放入队列等待业务处理
try{
var off = 0, total = 0;
while(off < req.body.length){
let decoded = type.decode(req.body, off); // No precision loss.
off = decoded.offset
state.addToQueue("track_queue", {item:decoded.value})
total++;
}

    }catch(e){
        console.log(e)
    }</pre>

标题:NodeJs如何解析avro二进制数据
作者:hugh0524
地址:https://blog.uproject.cn/articles/2019/05/14/1557806617681.html