Mdthbs

I want count the response time max and average from nginx logs, based on hourly or minutes per api...

nginx.log sample:

10.1.1.1 - - [25/Aug/2019:05:26:30 +0700] "POST /v2/api/find/outlet/ HTTP/1.1" 200 2667 "-" "okhttp/3.12.0" "118.215.153.47" 0.178 0.178 .

10.1.1.1 - - [25/Aug/2019:05:26:30 +0700] "POST /v2/api/find/outlet/ HTTP/1.1" 200 2847 "-" "okhttp/3.12.0" "189.246.151.188" 0.177 0.178 .

10.1.1.1 - - [25/Aug/2019:05:27:52 +0700] "GET /v2/api/menu/category HTTP/1.1" 401 40 "-" "okhttp/3.12.0" "139.194.84.246" 0.007 0.007 .

10.1.1.1 - - [25/Aug/2019:05:27:52 +0700] "GET /v2/api/user/point HTTP/1.1" 200 152 "-" "okhttp/3.12.0" "202.80.217.172" 0.028 0.028 .

10.1.1.1 - - [25/Aug/2019:05:27:52 +0700] "GET /v2/api/user/destination HTTP/1.1" 200 169 "-" "okhttp/3.12.0" "36.91.42.35" 0.019 0.019 .

10.1.1.1 - - [25/Aug/2019:05:28:52 +0700] "POST /v2/api/transaction/inquiry HTTP/1.1" 200 503 "-" "okhttp/3.12.0" "36.89.234.129" 0.374 0.374 .

10.1.1.1 - - [25/Aug/2019:05:28:52 +0700] "POST /v2/api/transaction/confirm HTTP/1.1" 200 874 "-" "okhttp/3.12.0" "36.89.234.129" 0.394 0.394 .

10.1.1.1 - - [25/Aug/2019:05:28:52 +0700] "GET /v2/api/user/point HTTP/1.1" 200 152 "-" "okhttp/3.12.0" "114.5.147.117" 0.024 0.024 .

10.1.1.1 - - [25/Aug/2019:05:28:52 +0700] "GET /v2/api/menu/category HTTP/1.1" 403 40 "-" "okhttp/3.12.0" "139.194.84.246" 0.003 0.003 .

expectation sample below:

date                       |  api                               | max| avg



25/Aug/2019:05:26 /v2/api/find/outlet             2847 2757

25/Aug/2019:05:27 /v2/api/menu/category HTTP/1.1               1847 1757

25/Aug/2019:05:28 /v2/api/menu/category HTTP/1.1               1147 1257

I already try with this awk but only got the average:

awk '/25/Aug/2019:18/ {c++} END{print c}' access.log

Thanks

You really ought to use one of the many web server log file analysers (e.g. https://goaccess.io/ as suggested by @Bart. There's a decent summary of some alternatives at 7 Awesome Open Source Analytics Software For Linux and Unix, and google will find you more), but for a quick-and-dirty hack, you could use something like this:

awk -v OFS='t' '

    $0 ~ date { max[$7]+=$(NF-1); count[$7]++ };

    END {

      print "date","api","count","max","avg";

      for (i in max) {

        print date, i, count[i], max[i], max[i]/count[i] }

      }' 

  date="25/Aug/2019" nginx.log

Output based on your sample is (note, the fields are separated by tabs, not spaces):

date    api     count   max     avg

25/Aug/2019     /v2/api/find/outlet/    2       0.356   0.178

25/Aug/2019     /v2/api/user/destination        1       0.019   0.019

25/Aug/2019     /v2/api/transaction/inquiry     1       0.374   0.374

25/Aug/2019     /v2/api/user/point      2       0.052   0.026

25/Aug/2019     /v2/api/transaction/confirm     1       0.394   0.394

25/Aug/2019     /v2/api/menu/category   2       0.01    0.005

BTW, the awk script above is based on the assumption that the response time for a given request is in the 2nd last field ($(NF-1)). I've had to guess here because you haven't told us what logfile format you have configured for your nginx server, or what the last few fields on each line are.