如何在golang中正确使用judy array lib？

Question

In golang, the way calling C library is different from what's used in other mainframe dynamic language like PHP / Python / Java because Golang has a different multitasking mechanism which is not OS thread based, so call c function may result in a context switching or thread switching as I understand. In my project I'm trying to use Judy Array in Golang (as a queue worker) to do some simple but large amount dict-related calculation like "select distinct", so

What's the best practice to involve such c lib (for relatively high density calculation) and minimalise the performance overhead introduced as much as possible?

Answer 1

Despite the title, the question here really has two parts: a generic one about golang and C-interfacing for efficiency, and a specific one about performant use of judy arrays.

This thread seems to summarize the costs: https://groups.google.com/forum/#!topic/golang-nuts/RTtMsgZi88Q , so yeah its expensive compared to straight C, and you should try to minimize the crossover points from Go to C.

Here's additional, judy array specific advice: I've used judy arrays before in C/C++ code. The library's interface is not intuitive in certain places. And by default it uses a C-macro based API, which makes it tricky to get the interface usage correct because the compiler can't offer as much help as usual.

What I recommend, therefore, is that you write your tests and benchmarks in C first, so you understand the API and its weird cases. Judy arrays when benchmarked for my application (vs C++ vector of strings) were 3x faster, so it can be worth it. But break the task into three phases. First do what you want to do in C, and make sure it works as expected in your own C code. Then expand the basic C interface to handle batches of what you need done, so as to minimize the number of Go->C switches. Then bind your new C interface from Go.