V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
推荐学习书目
Learn Python the Hard Way
Python Sites
PyPI - Python Package Index
http://diveintopython.org/toc/index.html
Pocoo
值得关注的项目
PyPy
Celery
Jinja2
Read the Docs
gevent
pyenv
virtualenv
Stackless Python
Beautiful Soup
结巴中文分词
Green Unicorn
Sentry
Shovel
Pyflakes
pytest
Python 编程
pep8 Checker
Styles
PEP 8
Google Python Style Guide
Code Style from The Hitchhiker's Guide
bestehen
V2EX  ›  Python

为啥第一台电脑不能返回数据,第二台电脑 ,第一台电脑被反爬后不能返回数据

  •  
  •   bestehen · 2018-06-19 14:08:21 +08:00 · 1793 次点击
    这是一个创建于 2350 天前的主题,其中的信息可能已经有所发展或是发生改变。
    第一个:
    curl -v 'https://www.qichacha.com/gongsi_getList' -H 'cookie: acw_tc=AQAAADKhg2r1fgQAzB38csKGgfa3ll5A; PHPSESSID=mr3rtla2pree2kma06in109lp7; UM_distinctid=16409b0ebb924e-01a16023a76872-19336953-13c680-16409b0ebba682; zg_did=%7B%22did%22%3A%20%2216409b0ebcf400-0a47e9ae97aea3-19336953-13c680-16409b0ebd0661%22%7D; _uab_collina=152917094889257593834626; _umdata=535523100CBE37C3B9E8426803FAE682F695DD5C372880100D01308BEF2CB953FEF5024D24D0BA85CD43AD3E795C914C6E418FBD7FCF11CFC02159EA6BDBD805; hasShow=1; zg_de1d1a35bfa24ce29bbf2c7eb17e6c4f=%7B%22sid%22%3A%201529381569060%2C%22updated%22%3A%201529381569062%2C%22info%22%3A%201529170947031%2C%22superProperty%22%3A%20%22%7B%7D%22%2C%22platform%22%3A%20%22%7B%7D%22%2C%22utm%22%3A%20%22%7B%7D%22%2C%22referrerDomain%22%3A%20%22www.baidu.com%22%2C%22cuid%22%3A%20%22b77823811d3a8fd207eef49092fcf4d6%22%7D; CNZZDATA1254842228=222723142-1529170913-https%253A%252F%252Fwww.qichacha.com%252F%7C1529376747; Hm_lvt_3456bee468c83cc63fb5147f119f1075=1529170947,1529201010,1529202769,1529381570; Hm_lpvt_3456bee468c83cc63fb5147f119f1075=1529381570' -H 'origin: https://www.qichacha.com' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.62 Safari/537.36' -H 'content-type: application/x-www-form-urlencoded; charset=UTF-8' -H 'accept: */*' -H 'referer: https://www.qichacha.com/' -H 'authority: www.qichacha.com' -H 'x-requested-with: XMLHttpRequest' --data $'key=\u767e\u5ea6&type=0' --compressed
    * About to connect() to www.qichacha.com port 443 (#0)
    * Trying 42.81.4.218...
    * Connected to www.qichacha.com (42.81.4.218) port 443 (#0)
    * Initializing NSS with certpath: sql:/etc/pki/nssdb
    * CAfile: /etc/pki/tls/certs/ca-bundle.crt
    CApath: none
    * SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
    * Server certificate:
    * subject: CN=*.qichacha.com,OU=IT,O=苏州企查查网络科技有限公司,L=苏州市,ST=江苏省,C=CN
    * start date: Jun 16 00:00:00 2017 GMT
    * expire date: Jun 15 23:59:59 2020 GMT
    * common name: *.qichacha.com
    * issuer: CN=GeoTrust SSL CA - G3,O=GeoTrust Inc.,C=US
    > POST /gongsi_getList HTTP/1.1
    > Host: www.qichacha.com
    > cookie: acw_tc=AQAAADKhg2r1fgQAzB38csKGgfa3ll5A; PHPSESSID=mr3rtla2pree2kma06in109lp7; UM_distinctid=16409b0ebb924e-01a16023a76872-19336953-13c680-16409b0ebba682; zg_did=%7B%22did%22%3A%20%2216409b0ebcf400-0a47e9ae97aea3-19336953-13c680-16409b0ebd0661%22%7D; _uab_collina=152917094889257593834626; _umdata=535523100CBE37C3B9E8426803FAE682F695DD5C372880100D01308BEF2CB953FEF5024D24D0BA85CD43AD3E795C914C6E418FBD7FCF11CFC02159EA6BDBD805; hasShow=1; zg_de1d1a35bfa24ce29bbf2c7eb17e6c4f=%7B%22sid%22%3A%201529381569060%2C%22updated%22%3A%201529381569062%2C%22info%22%3A%201529170947031%2C%22superProperty%22%3A%20%22%7B%7D%22%2C%22platform%22%3A%20%22%7B%7D%22%2C%22utm%22%3A%20%22%7B%7D%22%2C%22referrerDomain%22%3A%20%22www.baidu.com%22%2C%22cuid%22%3A%20%22b77823811d3a8fd207eef49092fcf4d6%22%7D; CNZZDATA1254842228=222723142-1529170913-https%253A%252F%252Fwww.qichacha.com%252F%7C1529376747; Hm_lvt_3456bee468c83cc63fb5147f119f1075=1529170947,1529201010,1529202769,1529381570; Hm_lpvt_3456bee468c83cc63fb5147f119f1075=1529381570
    > origin: https://www.qichacha.com
    > accept-encoding: gzip, deflate, br
    > accept-language: en-US,en;q=0.9
    > user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.62 Safari/537.36
    > content-type: application/x-www-form-urlencoded; charset=UTF-8
    > accept: */*
    > referer: https://www.qichacha.com/
    > authority: www.qichacha.com
    > x-requested-with: XMLHttpRequest
    > Content-Length: 17
    >
    * upload completely sent off: 17 out of 17 bytes
    < HTTP/1.1 200 OK
    < Server: Tengine
    < Content-Type: text/html; charset=UTF-8
    < Transfer-Encoding: chunked
    < Connection: keep-alive
    < Date: Tue, 19 Jun 2018 05:53:51 GMT
    < Vary: Accept-Encoding
    < Expires: Thu, 19 Nov 1981 08:52:00 GMT
    < Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
    < Pragma: no-cache
    < Content-Encoding: gzip
    < Via: cache19.l2nu20-3[116,200-0,M], cache40.l2nu20-3[117,0], cache8.cn247[132,200-0,M], cache8.cn247[133,0]
    < X-Cache: MISS TCP_MISS dirn:-2:-2 mlen:-1
    < X-Swift-SaveTime: Tue, 19 Jun 2018 05:53:52 GMT
    < X-Swift-CacheTime: 0
    < Timing-Allow-Origin: *
    < EagleId: 2a51048815293876318761329e


    第二个

    curl -v 'https://www.qichacha.com/gongsi_getList' -H 'cookie: acw_tc=AQAAADKhg2r1fgQAzB38csKGgfa3ll5A; PHPSESSID=mr3rtla2pree2kma06in109lp7; UM_distinctid=16409b0ebb924e-01a16023a76872-19336953-13c680-16409b0ebba682; zg_did=%7B%22did%22%3A%20%2216409b0ebcf400-0a47e9ae97aea3-19336953-13c680-16409b0ebd0661%22%7D; _uab_collina=152917094889257593834626; _umdata=535523100CBE37C3B9E8426803FAE682F695DD5C372880100D01308BEF2CB953FEF5024D24D0BA85CD43AD3E795C914C6E418FBD7FCF11CFC02159EA6BDBD805; hasShow=1; Hm_lvt_3456bee468c83cc63fb5147f119f1075=1529170947,1529201010,1529202769,1529381570; CNZZDATA1254842228=222723142-1529170913-https%253A%252F%252Fwww.qichacha.com%252F%7C1529382147; zg_de1d1a35bfa24ce29bbf2c7eb17e6c4f=%7B%22sid%22%3A%201529386744674%2C%22updated%22%3A%201529386772675%2C%22info%22%3A%201529170947031%2C%22superProperty%22%3A%20%22%7B%7D%22%2C%22platform%22%3A%20%22%7B%7D%22%2C%22utm%22%3A%20%22%7B%7D%22%2C%22referrerDomain%22%3A%20%22%22%2C%22cuid%22%3A%20%22b77823811d3a8fd207eef49092fcf4d6%22%7D; Hm_lpvt_3456bee468c83cc63fb5147f119f1075=1529386773' -H 'origin: https://www.qichacha.com' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.62 Safari/537.36' -H 'content-type: application/x-www-form-urlencoded; charset=UTF-8' -H 'accept: */*' -H 'referer: https://www.qichacha.com/' -H 'authority: www.qichacha.com' -H 'x-requested-with: XMLHttpRequest' --data $'key=\u767e\u5ea6&type=0' --compressed
    * Trying 42.81.4.217...
    * TCP_NODELAY set
    * Connected to www.qichacha.com (42.81.4.217) port 443 (#0)
    * TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
    * Server certificate: *.qichacha.com
    * Server certificate: GeoTrust SSL CA - G3
    * Server certificate: GeoTrust Global CA
    > POST /gongsi_getList HTTP/1.1
    > Host: www.qichacha.com
    > cookie: acw_tc=AQAAADKhg2r1fgQAzB38csKGgfa3ll5A; PHPSESSID=mr3rtla2pree2kma06in109lp7; UM_distinctid=16409b0ebb924e-01a16023a76872-19336953-13c680-16409b0ebba682; zg_did=%7B%22did%22%3A%20%2216409b0ebcf400-0a47e9ae97aea3-19336953-13c680-16409b0ebd0661%22%7D; _uab_collina=152917094889257593834626; _umdata=535523100CBE37C3B9E8426803FAE682F695DD5C372880100D01308BEF2CB953FEF5024D24D0BA85CD43AD3E795C914C6E418FBD7FCF11CFC02159EA6BDBD805; hasShow=1; Hm_lvt_3456bee468c83cc63fb5147f119f1075=1529170947,1529201010,1529202769,1529381570; CNZZDATA1254842228=222723142-1529170913-https%253A%252F%252Fwww.qichacha.com%252F%7C1529382147; zg_de1d1a35bfa24ce29bbf2c7eb17e6c4f=%7B%22sid%22%3A%201529386744674%2C%22updated%22%3A%201529386772675%2C%22info%22%3A%201529170947031%2C%22superProperty%22%3A%20%22%7B%7D%22%2C%22platform%22%3A%20%22%7B%7D%22%2C%22utm%22%3A%20%22%7B%7D%22%2C%22referrerDomain%22%3A%20%22%22%2C%22cuid%22%3A%20%22b77823811d3a8fd207eef49092fcf4d6%22%7D; Hm_lpvt_3456bee468c83cc63fb5147f119f1075=1529386773
    > origin: https://www.qichacha.com
    > accept-encoding: gzip, deflate, br
    > accept-language: en-US,en;q=0.9
    > user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.62 Safari/537.36
    > content-type: application/x-www-form-urlencoded; charset=UTF-8
    > accept: */*
    > referer: https://www.qichacha.com/
    > authority: www.qichacha.com
    > x-requested-with: XMLHttpRequest
    > Content-Length: 17
    >
    * upload completely sent off: 17 out of 17 bytes
    < HTTP/1.1 200 OK
    < Server: Tengine
    < Content-Type: text/html; charset=UTF-8
    < Transfer-Encoding: chunked
    < Connection: keep-alive
    < Date: Tue, 19 Jun 2018 05:51:11 GMT
    < Vary: Accept-Encoding
    < Expires: Thu, 19 Nov 1981 08:52:00 GMT
    < Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
    < Pragma: no-cache
    < Content-Encoding: gzip
    < Via: cache19.l2em21-1[138,200-0,M], cache2.l2em21-1[139,0], cache8.cn247[169,200-0,M], cache5.cn247[170,0]
    < X-Cache: MISS TCP_MISS dirn:-2:-2 mlen:-1
    < X-Swift-SaveTime: Tue, 19 Jun 2018 05:51:11 GMT
    < X-Swift-CacheTime: 0
    < Timing-Allow-Origin: *
    < EagleId: 2a51048515293874710697396e
    <
    * Curl_http_done: called premature == 0
    * Connection #0 to host www.qichacha.com left intact
    [{"KeyNo":"3f603703d59a04cbe427e5825099a565","Name":"<em>\u767e\u5ea6<\/em>\u5728\u7ebf\u7f51\u7edc\u6280\u672f(\u5317\u4eac)\u6709\u9650\u516c\u53f8","Reason":"\u80a1\u7968\u7b80\u79f0","Value":"<em>\u767e\u5ea6<\/em>","OperName":null,"ImageUrl":null},{"KeyNo":"576c21e3468a6b178bbf291e4820e896","Name":"\u5317\u4eac<em>\u767e\u5ea6<\/em>\u7f51\u8baf\u79d1\u6280\u6709\u9650\u516c\u53f8","Reason":"\u516c\u53f8\u540d\u79f0","Value":"\u5317\u4eac<em>\u767e\u5ea6<\/em>\u7f51\u8baf\u79d1\u6280\u6709\u9650\u516c\u53f8","OperName":null,"ImageUrl":null},{"KeyNo":"040087950737026999780939d6a623e9","Name":"<em>\u767e\u5ea6<\/em>\u56fd\u9645\u79d1\u6280(\u6df1\u5733)\u6709\u9650\u516c\u53f8","Reason":"\u516c\u53f8\u540d\u79f0","Value":"<em>\u767e\u5ea6<\/em>\u56fd\u9645\u79d1\u6280(\u6df1\u5733)\u6709\u9650\u516c\u53f8","OperName":null,"ImageUrl":null},{"KeyNo":"9459ee4a7789af50354b26dfc971c28a","Name":"<em>\u767e\u5ea6<\/em>\u79fb\u4fe1\u7f51\u7edc\u6280\u672f(\u5317\u4eac)\u6709\u9650\u516c\u53f8","Reason":"\u516c\u53f8\u540d\u79f0","Value":"<em>\u767e\u5ea6<\/em>\u79fb\u4fe1\u7f51\u7edc\u6280\u672f(\u5317\u4eac)\u6709\u9650\u516c\u53f8","OperName":null,"ImageUrl":null},{"KeyNo":"587d870f88a25bc849102850fcef9c0e","Name":"<em>\u767e\u5ea6<\/em>\u65f6\u4ee3\u7f51\u7edc\u6280\u672f(\u5317\u4eac)\u6709\u9650\u516c\u53f8","Reason":"\u516c\u53f8\u540d\u79f0","Value":"<em>\u767e\u5ea6<\/em>\u65f6\u4ee3\u7f51\u7edc\u6280\u672f(\u5317\u4eac)\u6709\u9650\u516c\u53f8","OperName":null,"ImageUrl":null}]%
    3 条回复    2018-06-22 01:21:47 +08:00
    woscaizi
        1
    woscaizi  
       2018-06-19 14:31:45 +08:00 via iPhone
    IP 被限制
    opengps
        2
    opengps  
       2018-06-20 08:01:51 +08:00 via Android
    这些爬虫起家的网站,都会有反爬虫策略的
    bestehen
        3
    bestehen  
    OP
       2018-06-22 01:21:47 +08:00
    @woscaizi 我用一样的 ip 不一样结果啊
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   实用小工具   ·   3694 人在线   最高记录 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 26ms · UTC 04:39 · PVG 12:39 · LAX 20:39 · JFK 23:39
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.