MongoDB记录可疑流量

Question

I'd like to keep a collection of suspicious traffic that resembles the following schema:

"_id": ObjectId(###),
"count": NumberInt(6),
"ip": NumberInt(2147483647),
"requests": {
[ "uri": "/path/to/something/",
  "last": NumberInt(1419023477)
  "count": NumberInt(2) ],
[ "uri": "/path/to/something/else/",
  "last": NumberInt(1419023478)
  "count": NumberInt(4) ]
}

Can somebody help me with a single upsert that will:

• add the request URI to the embedded document array
• increment the number of requests for that URI
• set the last request date for that URI
• and finally increment overall request count for that IP

Somebody asked me to post what I've got so far:

$db->coll->update(array('ip' => $ip),
                array('$addToSet' => array('req' => array('$set' => array('last'  => $timestamp),
                                                          '$inc' => array('count' => 1)))),
                array('upsert' => true)
                );

As you can see, it's not yet searching for the embedded doc with the matching uri ($uri)

Answer 1

I'd change your document structure. Naively, one IP will make requests to an ever-growing collection of URIs. Having arrays that grow without bound is not a good idea in MongoDB, and you will find it slow and cumbersome to deal with them. I'd suggest basing each document on a request, instead of on an IP, so the documents look like:

{
    "ip" : "192.168.1.1",
    "uri" : "/food/cookies/chocolatechip",
    "timestamp" : ISODate("2014-12-22T18:44:26.860Z")
}

I put in last in as a datetime, which is almost always what you should prefer for datetimes in MongoDB. I renamed it timestamp since it no longer makes sense to call it last. Also, why are you storing ip as a number? I'm not an expert on the rules for IP addresses, but I think that is dangerous as 192.168.1.1 is not the same IP address as 19.216.81.1 but both give the same number when you drop the .'s.

Now, you accomplish all of your objectives for a single upsert with one insert of a new document. You can use .count() to find counts and .sort() to find the most recent requests, and can index the necessary queries so these operations are fast.