Given a requirement where a large csv file (about 300 bytes long lines ending in /n) needs to be processed in a typical ETL: Extract, Transform, Load fashion (each line read, split and composition of a JSON inserted in a DB). Would it be beneficial to spawn one (or more) goroutines that worked together processing the file?
What would need to be done to create a bufio.Scanner
that started reading from a random position of the file?
将goroutines与缓冲读取结合使用可优化读取大文件
- 写回答
- 好问题 0 提建议
- 追加酬金
- 关注问题
- 邀请回答
-
1条回答 默认 最新
- drs7798 2016-02-05 04:59关注
Would it be beneficial to spawn one (or more goroutines)?
Yes, absolutely. In general, you could have 3 concurrent goroutines on each E, T, L, and have them coordinated via channels.
For more insights, check out this awesome talk from Rob Pike himself:
Concurrency is not Parallelism
: https://goo.gl/cp8xgFTalk Slides
http://talks.golang.org/2012/waza.slide#1本回答被题主选为最佳回答 , 对您是否有帮助呢?解决 无用评论 打赏 举报
悬赏问题
- ¥20 删除和修改功能无法调用
- ¥15 kafka topic 所有分副本数修改
- ¥15 小程序中fit格式等运动数据文件怎样实现可视化?(包含心率信息))
- ¥15 如何利用mmdetection3d中的get_flops.py文件计算fcos3d方法的flops?
- ¥40 串口调试助手打开串口后,keil5的代码就停止了
- ¥15 电脑最近经常蓝屏,求大家看看哪的问题
- ¥60 高价有偿求java辅导。工程量较大,价格你定,联系确定辅导后将采纳你的答案。希望能给出完整详细代码,并能解释回答我关于代码的疑问疑问,代码要求如下,联系我会发文档
- ¥50 C++五子棋AI程序编写
- ¥30 求安卓设备利用一个typeC接口,同时实现向pc一边投屏一边上传数据的解决方案。
- ¥15 SQL Server analysis services 服务安装失败