Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: wodegithubzh/ProxyPool
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: a0edcb0
Choose a base ref
...
head repository: Python3WebSpider/ProxyPool
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: cabcd96
Choose a head ref
  • 13 commits
  • 17 files changed
  • 6 contributors

Commits on Nov 23, 2023

  1. Python3WebSpider#140 及修改可环境变量配置的参数 (Python3WebSpider#202)

    * 1、设置scheduler.py中dev模式的flask run,不进行启动时的自动重新加载reload
    
    * 1、修改PROXY_SCORE_MAX,PROXY_SCORE_MIN,PROXY_SCORE_INIT三项配置,为可环境变量配置
    2、添加可环境变量配置项TEST_DONT_SET_MAX_SCORE,允许设置当tester检测到某个proxy可用时,只是保持原score,而不将其score设置成max。
    
    * 增加获取proxy接口的认证header, API-KEY。可配置,默认不需要
    inVains authored Nov 23, 2023
    Configuration menu
    Copy the full SHA
    0a6d861 View commit details
    Browse the repository at this point in the history

Commits on Dec 1, 2023

  1. Configuration menu
    Copy the full SHA
    2344ee1 View commit details
    Browse the repository at this point in the history

Commits on Dec 26, 2023

  1. Configuration menu
    Copy the full SHA
    97229e6 View commit details
    Browse the repository at this point in the history

Commits on Jan 1, 2024

  1. Configuration menu
    Copy the full SHA
    0033586 View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2024

  1. Configuration menu
    Copy the full SHA
    78b3244 View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2024

  1. validate testing ip

    Germey committed Jun 30, 2024
    Configuration menu
    Copy the full SHA
    7c77ad0 View commit details
    Browse the repository at this point in the history
  2. add proxy

    Germey committed Jun 30, 2024
    Configuration menu
    Copy the full SHA
    3ed558b View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2026

  1. Configuration menu
    Copy the full SHA
    dc32b04 View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2026

  1. fix: resolve core runtime bugs in tester, scheduler and proxy parsing (

    …Python3WebSpider#229)
    
    - utils/proxy: fix ValueError on multi-colon proxy strings in the str branch of convert_proxy_or_proxies, which made /random return 500 (Python3WebSpider#218)
    
    - scheduler: guard None process handles in finally so ENABLE_*=false no longer raises AttributeError (Python3WebSpider#62)
    
    - tester: replace deprecated get_event_loop()/asyncio.wait() with a fresh loop + asyncio.gather to fix 'Passing coroutines is forbidden' on Python 3.10+ (Python3WebSpider#191, Python3WebSpider#201)
    
    - tester: reuse one aiohttp session with a bounded connector instead of one per proxy, reducing memory growth and fd exhaustion (Python3WebSpider#119, Python3WebSpider#155)
    
    - tester: catch ClientResponseError/ContentTypeError + return_exceptions so bad httpbin responses no longer raise 'Task exception was never retrieved' (Python3WebSpider#208)
    Germey authored Jul 4, 2026
    Configuration menu
    Copy the full SHA
    91a1983 View commit details
    Browse the repository at this point in the history
  2. chore: modernize dependencies and container runtime (Python3WebSpider…

    …#230)
    
    - requirements: move to Flask 2.x and drop the ancient itsdangerous==0.24 / MarkupSafe<2.1 pins that blocked installs on Python 3.10+; bump redis to 4/5, aiohttp>=3.9, lxml<6, gevent<25; add explicit Werkzeug<3 bound
    
    - Dockerfile: base image python:3.7-alpine (EOL) -> python:3.11-slim so gevent/lxml/aiohttp install from wheels instead of compiling under musl, fixing common build failures and enabling arm64 (e.g. Raspberry Pi)
    
    Refs Python3WebSpider#167, Python3WebSpider#183, Python3WebSpider#192, Python3WebSpider#178, Python3WebSpider#149, Python3WebSpider#158
    Germey authored Jul 4, 2026
    Configuration menu
    Copy the full SHA
    c355a41 View commit details
    Browse the repository at this point in the history
  3. security: validate key query param and fix /count return type (Python…

    …3WebSpider#231)
    
    - server: validate the key query parameter against a strict charset (letters, digits, _ : -, max 64) so the API cannot be used to probe arbitrary redis keys; invalid keys return 400
    
    - server: /count without a key returned a bare int which Flask rejects; always return a string
    
    - README: document API_KEY and add a security note about exposing the pool publicly without authentication
    
    Refs Python3WebSpider#228
    Germey authored Jul 4, 2026
    Configuration menu
    Copy the full SHA
    40a326a View commit details
    Browse the repository at this point in the history
  4. feat: filter proxies by country via ?area=CN (Python3WebSpider#84, Py…

    …thon3WebSpider#177) (Python3WebSpider#233)
    
    * feat: support fetching multiple proxies via /random?count=N
    
    - redis: add RedisClient.randoms(count, ...) which prefers max-score proxies and falls back to rank, returning a de-duplicated random sample
    
    - server: /random accepts a count parameter and returns N random proxies (one per line); count<=1 keeps the original single-proxy behavior; works with key too
    
    - README: document the new count parameter
    
    Closes Python3WebSpider#194
    
    * feat: filter proxies by country via ?area=CN on /random and /all
    
    - utils/geo: add get_country_iso() backed by the bundled GeoLite2 db (previously an unused dependency); fails safe to None if unavailable
    
    - server: /random and /all accept an area parameter (country iso code) to return only proxies from that country; combines with count and key
    
    - README: document the area parameter
    
    Closes Python3WebSpider#84, Closes Python3WebSpider#177
    Germey authored Jul 4, 2026
    Configuration menu
    Copy the full SHA
    b915026 View commit details
    Browse the repository at this point in the history
  5. feat: make the tester anonymity-check URL configurable via TEST_ANONY…

    …MOUS_URL (Python3WebSpider#234)
    
    - setting: add TEST_ANONYMOUS_URL (default https://httpbin.org/ip)
    
    - tester: use TEST_ANONYMOUS_URL instead of the hardcoded httpbin.org endpoint so users can self-host httpbin and avoid public rate limits
    
    - README: document TEST_ANONYMOUS_URL and TEST_ANONYMOUS
    
    Closes Python3WebSpider#109
    Germey authored Jul 4, 2026
    Configuration menu
    Copy the full SHA
    cabcd96 View commit details
    Browse the repository at this point in the history
Loading