THE FORUM IS IN READ-ONLY MODE

This forum is in read-only mode. The new forum is live at https://discourse.openbullet.dev and registrations are open!

Help in Scraping DATA using OPENBULLET


  • Donator

    Hi guyz

    i'm scraping data using an API that not allows multiple requests at same time using same ip (Scraping is legal but there is limits)
    After the API response, bot should wait 1 sec before next request using same ip
    i found no bypass for this limit
    i set retry key, proxy will change after 1 retry, but i don't want make pressure on target with that unreasonable requests

    is there any way that i could set proxy status as Bad/Banned after Success/Custom/Fail status?
    i tried SET PROXY commands and OB/Config settings, i found no way


  • Admin

    BAN = retry data and ban the proxy
    RETRY = retry data with the same proxy
    Also you can use the Function > Delay block


  • Admin

    If you want to ban a proxy after SUCCESS you can do that in OB2, not in OB1


  • Donator

    @Ruri Can we create new custom STATUS in OB2, like we make custom wordlists using [Environment.ini]?
    i need a DONE instead of SUCCESS, or SUCCESS without saving Creds/Data in DB/TXT

    and also i don't wanna see Creds/Data and the captured data in the Runner UI / Or only see limited number of hits in runner

    both needed for data scraping and offline tools that i made using OB


  • Donator

    @Ruri said in Help in Scraping DATA using OPENBULLET:

    If you want to ban a proxy after SUCCESS you can do that in OB2, not in OB1

    No,
    Actually i wanna set proxy as BAD after SUCCESS or even FAIL, i don't want to ban that proxy


  • Admin

    Bad is treated as Banned, it's the same thing, I don't even know why I made a distinction.


  • Admin

    Anyways, about OB2 config settings

    75586ca5-d5bc-44f8-bc93-824ae2bd1bb2-immagine.png

    a4ff4a96-01b3-4a72-9c73-85748c4cbaa5-immagine.png

    and yes you can expand them if you edit the Environment.ini file


  • Donator

    @Ruri said in Help in Scraping DATA using OPENBULLET:

    Bad is treated as Banned, it's the same thing, I don't even know why I made a distinction.

    So u don't remember
    Bad proxy will go to active proxies after all proxies goes to BAD
    But BANNED proxies will only reactivated after all proxies banned

    with Retry (in KEYCHECK) we can set proxy as BAD and with BAN (in KEYCHECK) we can set proxy as BAN
    BAD is good to make proxies rotating, and BAN is good to deactivate dead proxies for long time
    two loops here works good
    and that's actually very useful


  • Donator

    @Ruri said in Help in Scraping DATA using OPENBULLET:

    Anyways, about OB2 config settings

    75586ca5-d5bc-44f8-bc93-824ae2bd1bb2-immagine.png

    a4ff4a96-01b3-4a72-9c73-85748c4cbaa5-immagine.png

    and yes you can expand them if you edit the Environment.ini file

    "Ban the proxy when the bot ends with one of these statuses" is good, but if you also add another option as "consider the proxy as Bad when the bot ends with one of these statuses" is better


Log in to reply