1. 首先,确保已经安装了`requests`和`beautifulsoup4`库。如果没有安装,可以使用以下命令安装:
bash
pip install requests beautifulsoup4
2. 接下来,使用以下代码来批量下载并保存网页内容:
python
import os
import requests
from bs4 import BeautifulSoup
def download_webpage(url, save_path):
response = requests.get(url)
soup = BeautifulSoup(response.text, '.parser')
filename = url.split('/')[-1] + '.'
with open(save_path, 'w', encoding='utf-8') as f:
f.write(soup.prettify())
def batch_download(urls, save_dir):
if not os.path.exists(save_dir):
os.makedirs(save_dir)
for url in urls:
save_path = os.path.join(save_dir, url.split('/')[-1])
download_webpage(url, save_path)
if __name__ == '__main__':
urls = [
'https://www.example1.com',
'https://www.example2.com',
'https://www.example3.com',
]
save_dir = 'downloaded_pages'
batch_download(urls, save_dir)
将上述代码保存为一个Python文件(例如:`batch_download.py`),然后在命令行中运行该文件。这将根据提供的URL列表下载所有页面,并将它们保存到名为`downloaded_pages`的文件夹中。