V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
推荐学习书目
Learn Python the Hard Way
Python Sites
PyPI - Python Package Index
http://diveintopython.org/toc/index.html
Pocoo
值得关注的项目
PyPy
Celery
Jinja2
Read the Docs
gevent
pyenv
virtualenv
Stackless Python
Beautiful Soup
结巴中文分词
Green Unicorn
Sentry
Shovel
Pyflakes
pytest
Python 编程
pep8 Checker
Styles
PEP 8
Google Python Style Guide
Code Style from The Hitchhiker's Guide
coolloves
V2EX  ›  Python

请教一个二维数组的数据处理.

  •  
  •   coolloves · 2019-02-25 16:51:37 +08:00 · 1446 次点击
    这是一个创建于 1859 天前的主题,其中的信息可能已经有所发展或是发生改变。
    分析 nginx 的日志,目前把把数据按照需求,处理成[(requesturl1,responsetime1),(requesturl2,responsetime2),.......].
    现在想要统计下 url 的次数,以及 avg responsetime,并排序.
    我能想到的就是 for 循环,感觉有点 low,请教大家一下更好的办法,多谢!
    
    
    a =[["d",2],["c",5],["a",9],["b",4],["b",2],["c",9]]
    uniqs = list(set([x[0] for x in a]))
    res = {}
    for i in uniqs:
        count = 0
        sumtime = 0
        for j in a:
            if i == j[0]:
                count = count + 1
                sumtime = sumtime + j[1]
            res[i] = [count,sumtime]
    lists = []
    for i in res.keys():
        lists.append((i,res[i][0],res[i][1]/res[i][0]))
    print(sorted(lists,key=lambda x:x[1],reverse=True))
    
    
    8 条回复    2019-02-28 14:16:02 +08:00
    chenstack
        1
    chenstack  
       2019-02-25 21:42:02 +08:00   ❤️ 1
    稍微减少了几行
    from itertools import groupby
    from operator import itemgetter

    a = [["d", 2], ["c", 5], ["a", 9], ["b", 4], ["b", 2], ["c", 9]]

    a = sorted(a, key=itemgetter(0))
    lists = []

    for key, group in groupby(a, itemgetter(0)):
        time_list = [item[1] for item in group]
        lists.append((key, len(time_list), sum(time_list) / len(time_list)))

    print(sorted(lists, key=itemgetter(1), reverse=True))
    zzz686970
        2
    zzz686970  
       2019-02-25 22:25:43 +08:00   ❤️ 1
    ```py
    import collections
    a = [["d", 2], ["c", 5], ["a", 9], ["b", 4], ["b", 2], ["c", 9]]
    res = collections.defaultdict(list)
    x, y = zip(*a)
    for i in range(len(x)):
    res[x[i]] += y[i],

    print(sorted([(key, len(value), sum(value)/len(value)) for key, value in res.items()] , key = lambda x: (x[1]), reverse = True))
    ```
    试了一下 collections
    necomancer
        3
    necomancer  
       2019-02-26 13:26:26 +08:00   ❤️ 1
    import pandas as pd
    a =[["d",2],["c",5],["a",9],["b",4],["b",2],["c",9]]
    d = pd.DataFrame(a)
    [ (_[0], _[1].mean().get_values()) for _ in d.groupby(0)]

    Out:

    [('a', array([9.])),
    ('b', array([3.])),
    ('c', array([7.])),
    ('d', array([2.]))]
    chenstack
        4
    chenstack  
       2019-02-26 14:33:26 +08:00   ❤️ 1
    似乎还能更短, 要用 Python3
    from itertools import groupby
    from operator import itemgetter

    a = [["d", 2], ["c", 5], ["a", 9], ["b", 4], ["b", 2], ["c", 9]]

    print(sorted([(key, len(group), sum(item[1] for item in group) / len(group))
        for key, (*group,) in groupby(sorted(a, key=itemgetter(0)), itemgetter(0))], key=itemgetter(1), reverse=True))
    coolloves
        5
    coolloves  
    OP
       2019-02-26 15:52:53 +08:00
    @necomancer 请教,pd 这个没有次数统计.如何统计呢,我也在看 pandas
    necomancer
        6
    necomancer  
       2019-02-26 16:06:31 +08:00
    @coolloves groupby 不就是次数统计了么……你想实现什么的计数? pandas 有 unique 和 nunique 方法。
    princelai
        7
    princelai  
       2019-02-28 10:41:39 +08:00   ❤️ 1
    ```
    import pandas as pd
    a =[["d",2],["c",5],["a",9],["b",4],["b",2],["c",9]]
    df = pd.DataFrame(a)
    df.groupby(0).agg(["count","mean"])
    ```

    Out[5]:
    1
    count mean
    0
    a 1 9
    b 2 3
    c 2 7
    d 1 2
    coolloves
        8
    coolloves  
    OP
       2019-02-28 14:16:02 +08:00
    @princelai 谢谢,还是 pd 最简洁
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   我们的愿景   ·   实用小工具   ·   2427 人在线   最高记录 6543   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 32ms · UTC 16:00 · PVG 00:00 · LAX 09:00 · JFK 12:00
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.