用php将文件大量插入couchbase数据库 - 如何?

I am experimenting actually a little bit with couchbase server.

I have tried to read a mysql database table, build a document from each data row and then inserting the document with an id which I generate with

uniqid('table_name'); 

via cUrl, method is POST.

This so far works pretty good, until the script has inserted roundabout 7050 documents. Then an exception is thrown -> "No buffer space".

Until now I was not able to fix this, so I decided to collect i.e. 50 rows of data build a json_encode(d) string and POST it again via cUrl.

This worked so far if I don't set the id - but I can't figure out how to set the id of the inserted documents.

Actually I try to send my documents in a format like this:

{"docs": {
   "_id": {
          "geodata_de_54476f7e6adc57.14196038": {
              "table": "geodata_de",
              "country": "DE",
              "postal_code": "01945",
              "place_name": "Lindenau",
              "state_name": "Brandenburg",
              "state_code": "BB",
              "province_name": "",
              "province_code": "00",
              "community_name": "Oberspreewald-Lausitz",
              "community_code": "12066",
              "lat": "51.4",
              "lng": "13.7333",
              "Xco": "3861.1",
              "Yco": "943.614",
              "Zco": "4979.07"
           } 
       },  ...
   }

}

but this just inserts ONE document with the above object.

Maybe there is someone here who can point me the right direction.

douruyun8153
douruyun8153 我将看一下PHPSDK。谢谢!
接近 6 年之前 回复
douluo3256
douluo3256 您应该使用couchbasePHPSDK来插入文档,而不是通过curl来完成。
接近 6 年之前 回复

1个回答



我会使用 Couchbase PHP SDK 插入这些文档而不是使用curl。 http://docs.couchbase.com/developer/php-2.0/storing.html </ p>

对于CB,您不必在文档中设置ID。 这取决于。 我可能会看一下使用您的示例中的ID(“geodata_de_54476f7e6adc57.14196038”)并将其作为Couchbase中对象的键。 那你不一定需要_id。 Couchbase中的密钥最多可以包含250个字节的数据,您可以使它对您的应用程序有意义,这样您就可以非常快速地按键进行查找。</ p>

另一个选择是,如果你写了你的 文件系统的文档,你也可以使用专门用于批量加载文档的cbdocloader实用程序。 如果您使用的是linux,请访问/opt/couchbase/bin/tools/cbdocloader.


</ div>

展开原文

原文

I would use the Couchbase PHP SDK to insert these documents instead of using curl. http://docs.couchbase.com/developer/php-2.0/storing.html

Also for CB, you do not have to set the ID in the document itself. it depends. I might take a look at instead using the ID you have in your example ("geodata_de_54476f7e6adc57.14196038") and put it as the key for the object in Couchbase. Then you do not necessarily need the _id. The key in Couchbase can be up to 250 bytes of data and you can make it meaningful to your application so you can do lookup by key extremely fast.

Another option is, if you wrote your docs to the filesystem, you could also use cbdocloader utility which is specifically for bulk loading docs. If you are on linux it is in /opt/couchbase/bin/tools/cbdocloader.

dongshang3309
dongshang3309 我将看一下PHP SDK。 是的,我知道,没有必要将id存储在文档本身中。 我试图插入一个包含Id命名的doc对象的对象“docs”。 我的想法是上传一个sql文件并将每一行作为文档插入,并将其用作我的admincp中的一个函数。 所以我认为cbdocloader不适合我。 谢谢!
接近 6 年之前 回复
Csdn user default icon
上传中...
上传图片
插入图片
抄袭、复制答案,以达到刷声望分或其他目的的行为,在CSDN问答是严格禁止的,一经发现立刻封号。是时候展现真正的技术了!
立即提问
相关内容推荐