四虎小视频,欧美在线观看禁18,日韩理伦片在线观看视频播放

開(kāi)發(fā)環(huán)境

解釋器版本: python 3.8

代碼編輯器: pycharm 2021.2

第三方模塊

requests: pip install requests

csv

爬蟲(chóng)案例的步驟

1.確定url地址(鏈接地址)

2.發(fā)送網(wǎng)絡(luò)請(qǐng)求

3.數(shù)據(jù)解析(篩選數(shù)據(jù))

4.數(shù)據(jù)的保存(數(shù)據(jù)庫(kù)(mysql\mongodb\redis), 本地文件)

爬蟲(chóng)程序全部代碼

分析網(wǎng)頁(yè)

打開(kāi)開(kāi)發(fā)者工具，搜索關(guān)鍵字，找到正確url

Python爬取股票交易數(shù)據(jù)并可視化展示

導(dǎo)入模塊

import requests     # 發(fā)送網(wǎng)絡(luò)請(qǐng)求
import csv

請(qǐng)求數(shù)據(jù)

url = f'https://xueqiu.com/service/v5/stock/screener/quote/list?page=1&size=30&order=desc&order_by=amount&exchange=CN&market=CN&type=sha&_=1637908787379'
# 偽裝
headers = {
  # 瀏覽器偽裝
  'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36'
}
response = requests.get(url, headers=headers)
json_data = response.json()

解析數(shù)據(jù)

data_list = json_data['data']['list']
for data in data_list:
  data1 = data['symbol']
  data2 = data['name']
  data3 = data['current']
  data4 = data['chg']
  data5 = data['percent']
  data6 = data['current_year_percent']
  data7 = data['volume']
  data8 = data['amount']
  data9 = data['turnover_rate']
  data10 = data['pe_ttm']
  data11 = data['dividend_yield']
  data12 = data['market_capital']
  print(data1, data2, data3, data4, data5, data6, data7, data8, data9, data10, data11, data12)
  data_dict = {
      '股票代碼': data1,
      '股票名稱': data2,
      '當(dāng)前價(jià)': data3,
      '漲跌額': data4,
      '漲跌幅': data5,
      '年初至今': data6,
      '成交量': data7,
      '成交額': data8,
      '換手率': data9,
      '市盈率(TTM)': data10,
      '股息率': data11,
      '市值': data12,
  }
  csv_write.writerow(data_dict)

翻頁(yè)

對(duì)比1、2、3頁(yè)數(shù)據(jù)url，找到規(guī)律

Python爬取股票交易數(shù)據(jù)并可視化展示

for page in range(1, 56):
  url = f'https://xueqiu.com/service/v5/stock/screener/quote/list?page={page}&size=30&order=desc&order_by=amount&exchange=CN&market=CN&type=sha&_=1637908787379'

保存數(shù)據(jù)

file = open('data2.csv', mode='a', encoding='utf-8', newline='')
csv_write = csv.DictWriter(file, fieldnames=['股票代碼','股票名稱','當(dāng)前價(jià)','漲跌額','漲跌幅','年初至今','成交量','成交額','換手率','市盈率(TTM)','股息率','市值'])
csv_write.writeheader()
file.close()

實(shí)現(xiàn)效果

Python爬取股票交易數(shù)據(jù)并可視化展示

數(shù)據(jù)可視化全部代碼

導(dǎo)入數(shù)據(jù)

import pandas as pd
from pyecharts import options as opts
from pyecharts.charts import Bar

讀取數(shù)據(jù)

data_df = pd.read_csv('data2.csv')
df = data_df.dropna()
df1 = df[['股票名稱', '成交量']]
df2 = df1.iloc[:20]
print(df2['股票名稱'].values)
print(df2['成交量'].values)

可視化圖表

c = (
  Bar()
      .add_xaxis(list(df2['股票名稱']))
      .add_yaxis("股票成交量情況", list(df2['成交量']))
      .set_global_opts(
      title_opts=opts.TitleOpts(title="成交量圖表 - Volume chart"),
      datazoom_opts=opts.DataZoomOpts(),
  )
      .render("data.html")
)

print('數(shù)據(jù)可視化結(jié)果完成,請(qǐng)?jiān)诋?dāng)前目錄下查找打開(kāi) data.html 文件!')