正则表达式不匹配

Question

I tried to get SQL information from log files using regular expressions

The information I need is the running time of the SQL, the SQL statement, the parameters of the SQL

Here's my code:

package main

import (
    "os"
    "fmt"
    "io/ioutil"
    "regexp"
)

func main(){
    file,err:=os.OpenFile("./log.txt",os.O_RDWR,0755)
    if err !=nil{
        fmt.Println(err)
    }
    b,err:=ioutil.ReadAll(file)
    str:=string(b)
    r,err:=regexp.Compile(`\[ORM\][\w\W]+?(\d*.\d*ms|\d*.\d*s)\][ -]+\[([\w\W]+?)\]`)

    s:=r.FindAllStringSubmatch(str,-1)
    fmt.Println(s[0][3])
    fmt.Println(s[1][3])
}

This is my log sample

[ORM]2018/08/03 10:23:50 -[Queries/read] - [ OK / db.Query / 432.4ms] - [SELECT acc.*,gp.group_name,gp.group_id,org.org_name,group_concat(r.role_name) role_name FROM sys_account acc LEFT JOIN sys_org org on org.org_id=acc.org_id LEFT JOIN sys_group gp on gp.group_id=org.group_id LEFT JOIN sys_account_role ar on ar.acct_id=acc.acct_id and ar.is_del=0 LEFT JOIN sys_role r on r.role_id=ar.role_id where 1=1 and acc.acct_type=1 group by acc.acct_id order by acc.create_time desc LIMIT 0, 15] - `1` `ASDFASDF` nsq consumer2: INF 13 [RYOLST_Ch_admin/crm] (192.168.1.233:4150) received CLOSE_WAIT from nsqd nsq consumer2: INF 13 [RYOLST_Ch_admin/crm] (192.168.1.233:4150) beginning close nsq consumer2: INF 13 [RYOLST_Ch_admin/crm] (192.168.1.233:4150) readLoop exiting nsq consumer2: INF 13 [RYOLST_Ch_admin/crm] (192.168.1.233:4150) breaking out of writeLoop nsq consumer2: INF 13 [RYOLST_Ch_admin/crm] (192.168.1.233:4150) writeLoop exiting [ORM]2018/08/03 10:23:50 -[Queries/default] - [ OK / db.Query / 0.6ms] - [select * from sys_group where group_id=? ] - `111` `qwqwqw`

I hope so Print out the

`1` `ASDFASDF`

`111` `qwqwqw`

I'm now left with the last argument that I can't get

These parameters may or may not be multiple、You can also have too many arguments wrapping around the file

I have tried this myself：

r,err:=regexp.Compile(`\[ORM\][\w\W]+?(\d*.\d*ms|\d*.\d*s)\][ -]+\[([\w\W]+?)\][ -]*((\W\w*\W{1,2})*)`)
r,err:=regexp.Compile(`\[ORM\][\w\W]+?(\d*.\d*ms|\d*.\d*s)\][ -]+\[([\w\W]+?)\][- ]*([^
]*)`)

Answer 1

You probably want something like this regexp:

var re = regexp.MustCompile(`\[ORM\][^-]* -\[[^]]*\] - \[ .* / .* / ([^]]*)] - \[([^]]*)\] - (.*)`)

https://play.golang.org/p/MUBzaiyyCWt

This assumes well-formed data, one entry per line. If not you probably won't be able to parse it with a regexp, and even this makes some assumptions about queries not including the strings [] (which you might get away with).

First I'd double check that you absolutely cannot get this data output in the right format in the first place - if you can it'd be far better to get the output you want directly, perhaps even have a separate log, or even better send metrics like this to a time series database for later analysis. Parsing logs for this sort of info isn't great, particularly if done on an ongoing basis.

If you must parse logs, some advice on testing this:

• Use play.golang.org for a quick tests
• Use regexp.MustCompile to check your regexp works
• Create a function to parse the lines
• Write some tests which exercise your function, using lines from the log