dream5694 2013-07-07 12:14
浏览 15

关于大型项目速度的基本思路 - 每秒线程数,a.s.o

Hi Stackoverflowianers,

the following stack overflow belongs to my brain, not to my program:

I'm developing on a large scale project and now have to push millions of data requests in its database. For that to work I have to start a script which takes out a search text line by line from a .csv file and starts the process... this search word is sent to 6 official apis and gets data back inbetween 0,2 seconds and max. 10 seconds. as one by one is comming back, the script writes the response to the database.

I hope you understand what I mean. So my question is:

In the moment I do it with a cron job every minute... Is it possible to do it without any delay between the api-requests and still have a reliable and constant result?

Or will it be like stopping one process before it is ready because the next wants to take place and so on and so on...?

Thank you in advance for your thoughts on this.

Best regards, AceLine

PS: I do not have the experience to understand what happens at this speed of threads...

  • 写回答

1条回答 默认 最新

  • dongtangyi8962 2013-07-07 12:18
    关注

    I'm not sure I understood you, but if I did, it seems what you are missing is indexing your data.
    You index the data when you get it inputed to the system. After that, searches are much faster.
    The best way I know to index that is to use SOLR (Lucene).

    评论

报告相同问题?

悬赏问题

  • ¥20 软件测试决策法疑问求解答
  • ¥15 win11 23H2删除推荐的项目,支持注册表等
  • ¥15 matlab 用yalmip搭建模型,cplex求解,线性化处理的方法
  • ¥15 qt6.6.3 基于百度云的语音识别 不会改
  • ¥15 关于#目标检测#的问题:大概就是类似后台自动检测某下架商品的库存,在他监测到该商品上架并且可以购买的瞬间点击立即购买下单
  • ¥15 神经网络怎么把隐含层变量融合到损失函数中?
  • ¥15 lingo18勾选global solver求解使用的算法
  • ¥15 全部备份安卓app数据包括密码,可以复制到另一手机上运行
  • ¥20 测距传感器数据手册i2c
  • ¥15 RPA正常跑,cmd输入cookies跑不出来