THE FORUM IS IN READ-ONLY MODE

This forum is in read-only mode. The new forum is live at https://discourse.openbullet.dev and registrations are open!

OpenBullet 2 Implementation



  • Hello members,
    first of all i want to say I am not here to criticize or trash the work of the auther of openbullet
    i think he did an amazing job .. and for that i want to thank him.

    I just want to rumble a few words for learning purposes .

    I want to ask if the auther of OB removed the abortable background worker as the heart of OpenBullet in the upcoming version .

    because in as I think he already knows, that this implementation hurts performance …
    if you want you can see the number of threads running in (Resource monitor ) form the task manager..

    for example lets say I start 200 bots(threads in this case .. but it is actually more as we will see ) and wait for a couple of minutes you will see that the number of threads that OP consumes is well above 200, in my case it was 513 threads … not even the system consumes this much threads !.

    This on a desktop machine with 4 cores 8 threads will be very slow... IDK maybe good on sever machine with many cores ? but for desktop this is pure evil.

    But why it does not consume 100% CPU for those threads?
    from a developer point of view lets analyze what is happening...
    the first time you run the application the .NET CLR will assign a thread pool for this application.
    lets say with 10 threads (just a random number ) then we start the 200 background workers
    which as we know the workers run on the threadpool's threads.
    so the CLR sees the following (oh this application is doing IO that is not CPU intensive lets start more threads as all threads (the 10 in our example) are consumed , the CLR keeps starting new threads for the thread pool as these threads get blocked(YES BLOCKED) by the waiting for IO.
    final result is huge amount of threads for non CPU intensive task .
    leading to a lot of CPU context switching overhead and bad bad performance.

    So what is the solution?
    lets look at sentry MBA , it uses at its heart a component (HttpClient) that is derived from the open source library OverbyteICS this library was created in 1995 and still maintained till now.
    lets see what the author of that library has to say about asynchronous Networking :
    http://wiki.overbyte.eu/wiki/index.php/Asynchronous_Paradigm
    So TL;DR : No need for multiple threads to do networking IO. live example (high performance node.js servers use the same concept)

    OK what is your solution ?
    at first I developed an httpclient from scratch using OverBytesICS as an example..
    so OverbytesICS uses WSAAsyncSelect(); Function from the windows win32 winsock dll for Polling Non-blocking sockets using Windows messages and a Thread that Has a message loop.

    I did developed this (HtpClient-NonBlock) with C# successfully and then tested the results :
    it was very satisfying for around 500 bots.... remember single thread to control all instances of the HttpClient.

    but it turns out that the cap this model can handle is around 500 clients. thus I abandoned this model.
    I wanted the best performance I can get.
    on windows the highest performance networking is IO Completion Ports (IOCP) .
    so it turns out that SocketAsyncEventArgs class in .NET implements this Model For High Performance Servers.
    So i followed These Rules:
    1- Event based Programming Model.
    2- No async/await Junk.
    3-Use Unmanaged Memory and pointers in c# to achive highest performance and low GC rate
    3- use Span<T>.
    4- Reuse Objects as much as you can.

    So i combined non-blocking Socket IO with this model (in a cleaver way. XD) to come up with a new Httpclient that is really amazing performance , written in .NET 5 .
    I made a simple credential stuffing demo app to see real world performance at 300 hunderd bots
    with proxies ; I get between 5000 combos/min and 30,000 combos/min.
    I tested a 12,000,000 combo list in 6 hours !!!!!!!!!! using proxies ofc using 3000 bots.
    (support socks4/5/Http(connect) with/without authenticaton)
    Btw Ssl/TLs is handled by a Small C written library for ultra-fast Encoding and decoding.
    this client can MaxOut my bandwidth using a single socket.

    this library is very early alpha but it looks very promising.....

    the library is on github but private for now.... will release when it is fully finished.


  • Admin

    Hi, I moved away from the use of the BackgroundWorker and built a new library that I wrote from scratch. This said, the thread usage is still high. as I am allowing up to 1000 threads to be spawned from the system-managed hread pool at the moment.
    If you believe that you have the knowledge to improve this system, I will be waiting for a PR from you after open beta is released ^_^



  • thank you for your reply.
    I will try to help with this amazing project to my maximum effort.
    waiting eagerly for the release.
    best regards.


Log in to reply