问在多个线程中下载
EN

Stack Overflow用户

提问于 2017-03-19 20:35:27

回答 1查看 82关注 0票数 0

由于我在中国，使用YouTube对我来说是一个令人头疼的问题。我试着建立一个播放列表下载器(我知道youtube-dl，但想学习smth新的)，现在它基本上完成了它的工作。由于连接不稳定(中国的vpn)，在连接问题的情况下，浏览器只是停止下载，但不会将控制权交还给我的脚本。如何处理此异常？整个文件在这里，这是代码(从3d party服务下载)：github link：

def checkFileStatus(file_name):
"""Helper function for getDownloads, waits file to be downloaded"""
print('Ok, writing exactly ' + file_name)
checkFlag = False
while not checkFlag:
    if os.path.exists(file_name):
        print(file_name + " exists")
        check_one = os.path.getsize(file_name)
        time.sleep(5)
        check_two = os.path.getsize(file_name)
        if check_two == check_one:
            checkFlag = True
            print(file_name + ' has been saved')
            return
    else:
        time.sleep(5)


def getDownloads(clip_links, home_dir):
    """Manages the whole downloading process
    - opens webdrive with Chrome
    - saves and renames files from valid links
    -  quits webdrive"""
    chromeOptions = webdriver.ChromeOptions()
    prefs = {"download.default_directory": home_dir}
    chromeOptions.add_experimental_option("prefs", prefs)
    chromedriver = 'C:\Program Files (x86)\Google\Chrome\Application\chromedriver.exe'
    savemediaurl = 'http://savemedia.com/'
    for index, entry in enumerate(clip_links):
        driver = webdriver.Chrome(executable_path=chromedriver, chrome_options=chromeOptions)
        saved_file = downloadFromSavemedia(savemediaurl, driver, entry[1], index+1)
        if saved_file:
            old_name = entry[0] + '.mp4'
            checkFileStatus(old_name)
            new_name = str(index+1).zfill(2) + '. ' + entry[0] + '.mp4'
            os.rename(old_name, new_name)
        driver.quit()
    return

google-chrome

selenium

python

CDN&音视频通信出海专场

提供游戏出海、社交娱乐等方案，助力企业抢占出海市场

回答 1

Stack Overflow用户

发布于 2017-03-20 14:16:51

如果我正确理解了您的问题，您的下载完全停止，没有将控制权返回给您的脚本，您需要一种方法来检测/处理这种情况？

您可以尝试混合使用timeout-decorator和retry包：

from retry import retry
from timeout_decorator import timeout, TimeoutError

# <...>

@retry(TimeoutError, tries=3)
@timeout(300)  # Wait 300 seconds before raising the timeout exception
def _download(chromedriver, savemediaurl, entry, index):
    driver = webdriver.Chrome(executable_path=chromedriver, chrome_options=chromeOptions)
    try:
        saved_file = downloadFromSavemedia(savemediaurl, driver, entry[1], index+1)
        if saved_file:
            old_name = entry[0] + '.mp4'
            checkFileStatus(old_name)
            new_name = str(index+1).zfill(2) + '. ' + entry[0] + '.mp4'
            os.rename(old_name, new_name)
    finally:
        driver.quit()

def getDownloads(clip_links, home_dir):
    """Manages the whole downloading process
    - opens webdrive with Chrome
    - saves and renames files from valid links
    -  quits webdrive"""
    chromeOptions = webdriver.ChromeOptions()
    prefs = {"download.default_directory": home_dir}
    chromeOptions.add_experimental_option("prefs", prefs)
    chromedriver = 'C:\Program Files (x86)\Google\Chrome\Application\chromedriver.exe'
    savemediaurl = 'http://savemedia.com/'
    for index, entry in enumerate(clip_links):
        _download(chromedriver, savemediaurl, entry, index)

    return