去反思元编程/模板

Question

I have some code that receives protobuf messages that is basically duplicated in a couple places so I want to put it into a library. The problem is that the exact protobuf message that's used is different for each set of code.

EDIT: And I don't have the flexiblity of restructuring them.

I'm not entirely sure this is possible to solve without duplicate code in go, but I did make an attempt (below). Am I doing something wrong, or is this not something that's not possible? (Note: This is stripped down code, in the real code the objects have lots of additional fields)

Example.proto:

package testmsg;

enum RepStatus {
    DONE_OK = 0;
    DONE_ERROR = 1;
}

message ReqHeader {
    optional int64 user_id = 1;
}

message RespHeader {
    optional RepStatus status = 1;
    optional string error_msg = 2;
}

message PostReq {
    optional ReqHeader header = 1;
    optional bytes post_data = 2;
}

message PostResp {
    optional RespHeader header = 1;
}

message StatusReq {
    optional ReqHeader header = 1;
    optional string id = 2;
}

message StatusRep {
    optional RespHeader header = 1;
    optional string status = 2;
}

mini-service/service.go:

package miniservice

import "reflect"
import "github.com/golang/protobuf/proto"
import "testmsg"

type MiniService struct {
    name string
    reqType reflect.Type
    repType reflect.Type
}

func NewService(name string, reqPort int, reqType proto.Message, repType proto.Message) *MiniService {
    ms := new(MiniService)
    ms.name = name
    ms.reqType = reflect.TypeOf(reqType)
    ms.repType = reflect.TypeOf(repType)
    return ms
}

func (ms *MiniService) Handler(msgs []string) (string) {
    resp := reflect.New(ms.repType.Elem())

    msg := msgs[0]
    req := reflect.New(ms.reqType.Elem())
    err := proto.Unmarshal([]byte(msg), req)
    //add some error handling, or just get set _

    resp.Header = &testmsg.RespHeader{}
    //Call handler function that is unique per service
    //the signature will be something like:
    //handleRequest(reqType, respType) & called like:
    //handleRequest(req, resp)
    resp.Header.Status = testmsg.RepStatus_DONE_OK.Enum()

    respMsg, _ := proto.Marshal(resp)
    return string(respMsg)
}

testservice.go:

package main
import "github.com/golang/protobuf/proto"
import "testmsg"
import "mylibs/mini-service"

func main() {
    //fake out a zmq message
    req := &testmsg.PostReq{}
    req.Header = &testmsg.ReqHeader{}
    req.Header.MessageId = proto.Int64(10)
    reqMsg, _ := proto.Marshal(req)
    reqMsgs := []string{string(reqMsg)}

    ms := miniservice.NewService("tester", 5555, testmsg.PostReq, testmsg.PostResp)
    //What will be called when receiving a zmq request
    resp := ms.Handler(reqMsgs)
    log.Info(resp)
}

When I try to compile I get errors like:

resp.Header undefined (type reflect.Value has no field or method Header)

cannot use resp (type reflect.Value) as type proto.Message in argument to proto.Marshal:
reflect.Value does not implement proto.Message (missing ProtoMessage method)

Which make complete sense since the resp isn't connected to ms.respType.

Answer 1

I ended up completely abandoning reflect. I could work on generic objects, but I could not pass them on to handlers. Not being able to do this made it not worth using a library, so it seemed like a bad approach.

I created a simple "template" that I could copy and drop in the protobuf message names into. I then used go generate to build the messages that I needed. This let me put special go generate comments in my code that specified the types - so even though there is templating, filling it in and using it is done in a single go file.

So I put the base template in src/mylibs/req-handlers/base.tmp.go. I wanted to keep .go as the extension for syntax highlighting. In that file, I had generic things like {{RequestProto}} that would get replaced.

This script defined a ReqHandler type using some template variables:

type ReqHandlerFunc func(req *testmsg.{{RequestProto}}, resp *testmsg.{{ResponseProto}}) error

And I created an object that reference the handler function:

func NewReqHandler(name string, handler ReqHandlerFunc) *ReqHandler {
    ...
    rh.handler = handler
    return rh
}

and later in the code I called the handler function where it was needed:

err = rh.handler(req, resp)

In the bin directory, I added this script, which copies the template, and uses sed to replace some keywords with words I can specify in go code:

#!/bin/bash

if [ "$#" -ne 3 ] && [ "$#" -ne 4 ]; then
    echo "Usage: build_handler (Package Name) (Request Proto Name) (Response Proto Name) [Logger Name]"
    exit 1
fi

LIB=$1
REQ=$2
REP=$3
PKG="${LIB//-/}"
if [ "$#" -ne 4 ]; then
    LOG=${PKG}
else
    LOG=$4
fi

HANDLERS_DIR=$(dirname "$0")/../src/mylibs/req-handlers

#Generate go code
mkdir -p ${HANDLERS_DIR}/${LIB}/
GEN_FILE=${HANDLERS_DIR}/${LIB}/${LIB}_handler.go
cp ${HANDLERS_DIR}/base.tmpl.go ${GEN_FILE}
sed -i"" -e "s/{{PackageName}}/${PKG}/g" ${GEN_FILE}
sed -i"" -e "s/{{LoggerName}}/${LOG}/g" ${GEN_FILE}
sed -i"" -e "s/{{RequestProto}}/${REQ}/g" ${GEN_FILE}
sed -i"" -e "s/{{ResponseProto}}/${REP}/g" ${GEN_FILE}

Finally to make use of it, testservice.go would then have something like:

//go:generate build_handler testservicelib PostReq PostResp
import "mylibs/req-handlers/testservicelib"

When I run go generate, build_handler is called, which creates the mylibs/req-handlers/testservicelib library, which has a request handler with the PostReq and PostResp types. So I create a handler function that will have those as inputs:

func handleRequest(req *testmsg.PostReq, resp *testmsg.PostResp) error {
    ...
}

and pass that to my generated library:

reqHandler := testservicelib.NewReqHandler("test", handleRequest)

And life is good.

To build, the Makefile needed an extra step. Both go generate and go build/install steps are needed:

go generate testservice
go install testservice

Note that the generate call will run all the //go:generate comments in testservice.go, so in some cases I have more than 1 handler created.