hugh 的个人博客

背景： 做监控系统时，需要接受kafaka中传输的监控数据，传输来源采用avro编码 + application/octet-stream 二进制传输

处理方法:

1. 选择合适的框架 avsc， api文档

npm install avsc

2. 具体处理

// 定义长度标识
var i =0;
if("application/octet-stream" == req.headers['content-type']) {
            i = req.body.length
        }
// 定义schema

const type = avro.Type.forSchema({

"namespace": "xxx",

"name": "Model",

"type": "record",

"fields": [

{

"name": "appId",

"type": "string"

}...]
// 分段解析数据， 放入队列等待业务处理

try{

var off = 0, total = 0;

while(off < req.body.length){

let decoded = type.decode(req.body, off); // No precision loss.

off = decoded.offset

state.addToQueue("track_queue", {item:decoded.value})

total++;

}
    }catch(e){
        console.log(e)
    }</pre>