(转载)运维日志排查记录

不知原文出自哪里，如果您是作者，请私信我补充来源链接。

前言

记录一些排查常见日志的命令,方法wiki,欢迎补充(Markdown 语法)。

常用命令

1. 查找关键词并统计行数

   cat 2015_7_25_test_access.log | grep "sqlmap" | wc -l

2. 删除含有匹配字符的行

   sed -i '/Indy Library/d' 2015_7_25_test_access.log

3. 查找所有日志中的关键词

   find ./ -name "*.log" |xargs grep "sqlmap" |wc -l

4. 获取特殊行(如id)并且排序统计

   cat cszl988.log | awk '{print $1}' | awk -F : '{print $2}' | sort -u | wc -l

5. 正则匹配内容(如提取ip)

   grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}"

6. 去重并统计数量

   tail 3.log | awk '{print $7}' | sort | uniq -c

7. 批量提取(全流量中)数据包并且过滤数据

   #!/bin/bash
   for file in ` ls $1 `
   do
   parse_pcap -vvb $file | grep -v "Host:" | grep -v "Cookie:" | grep -v "User-Agent:" | grep -v "Accept:" | grep -v "Accept:" | grep -v "Accept-Language:" | grep -v "Accept-Encoding:" | grep -v "Connection:" | grep -v "Content-Type:" | grep -v "Content-Length" | grep -v "Server"
   done

8. url 解码

   cat luban.log | grep sqlmap | awk '{print $7}' | xargs python -c 'import sys, urllib; print urllib.unquote(sys.argv[1])'

9. 欢迎补充….

示范:xxxx站注入日志排查

• 查看所有sqlmap注入记录条数

  [root@pentest temp]# cat luban.log | grep sqlmap | wc -l
  1241

• 预览几条url

  cat luban.log | grep sqlmap | awk '{print $7}' | more
  /news.php?id=771%28.%28%22%29.%27%29%29%27&fid=168
  /news.php?id=771%27IddP%3C%27%22%3EvCBw&fid=168
  /news.php?id=771%29%20AND%201148%3D8887%20AND%20%288975%3D8975&fid=168
  /news.php?id=771%29%20AND%208790%3D8790%20AND%20%287928%3D7928&fid=168
  /news.php?id=771%20AND%204294%3D9647&fid=168
  /news.php?id=771%20AND%208790%3D8790&fid=168
  /news.php?id=771%27%29%20AND%205983%3D7073%20AND%20%28%27UwRr%27%3D%27UwRr&fid=168
  /news.php?id=771%27%29%20AND%208790%3D8790%20AND%20%28%27hwaT%27%3D%27hwaT&fid=168
  /news.php?id=771%27%20AND%206578%3D7565%20AND%20%27EoTZ%27%3D%27EoTZ&fid=168
  /news.php?id=771%27%20AND%208790%3D8790%20AND%20%27lBdL%27%3D%27lBdL&fid=168
  /news.php?id=771%25%27%20AND%205177%3D1107%20AND%20%27%25%27%3D%27&fid=168
  /news.php?id=771%25%27%20AND%208790%3D8790%20AND%20%27%25%27%3D%27&fid=168

• 方便查看 urldecode

  cat luban.log | grep sqlmap | awk '{print $7}' | xargs python -c 'import sys, urllib; print urllib.unquote(sys.argv[1])'
  /news.php?id=771&fid=168
  /news.php?id=771&fid=168 AND ASCII(SUBSTRING((SELECT DISTINCT(COALESCE(CAST(schemaname AS CHARACTER(10000)),(CHR(32)))) FROM pg_tables OFFSET 1 LIMIT 1)::text FROM 3 FOR 1))>
  97
  /news.php?id=771&fid=168 UNION ALL SELECT NULL,(CHR(113)||CHR(122)||CHR(106)||CHR(120)||CHR(113))||(CHR(103)||CHR(75)||CHR(78)||CHR(87)||CHR(76)||CHR(74)||CHR(110)||CHR(1
  15)||CHR(100)||CHR(85))||(CHR(113)||CHR(122)||CHR(120)||CHR(113)||CHR(113)),NULL,NULL,NULL,NULL,NULL,NULL,NULL UNION ALL SELECT NULL,(CHR(113)||CHR(122)||CHR(106)||CHR(120)||CHR(113))||(CHR(113)||CHR(71)||C
  HR(74)||CHR(82)||CHR(101)||CHR(120)||CHR(69)||CHR(112)||CHR(117)||CHR(79))||(CHR(113)||CHR(122)||CHR(120)||CHR(113)||CHR(113)),NULL,NULL,NULL,NULL,NULL,NULL,NULL--