Validation of proxies

From Documentation
Jump to: navigation, search

The software has a multi-threaded proxy validator, it is used to make sure the proxies are good before using them for any task. Proxy validator runs automatically after you upload or add the proxies into the software. During validation, the software determines the type of each proxy (http or socks) and its external IP address. By default, the proxies are re-validated automatically every 24 hours to make sure that they are still good (you can change validation period in the settings).

Validation is useful to filter out bad proxies, but it cannot give a 100% guarantee that the remaining proxies are good. There are 2 common issues with proxies that cannot be reliably detected by the validator:

  1. Latency issues. If there is already a substantial amount of traffic to the proxy, you may be experiencing severe delays and timeouts when loading web pages through such proxy. While sometimes the page may load fine, at other times it may take minutes to load a page, or it may not load at all. This problem cannot be detected by the validator, because it contacts the proxy only once during the validation. If it happens to respond within a reasonable time frame, it is considered valid. The next minute it may become overwhelmed by the traffic, but the validator wouldn't know that.
  2. Stability issues. A proxy is nothing else but a computer connected to the Internet, that forwards your requests to other computers. If that computer goes off or looses Internet connection - the proxy goes off as well. So it happens often that the validator connects to the proxy and it responds fine, so it is marked as good. The next minute someone flips the switch on the computer where the proxy runs, and it is no longer available. So the software may show the proxy as good and validated, but in fact it may not be, because it went down after validation.

It is good to understand that the validator checks whether the proxy is good at the moment of validation and provides absolutely no guarantees that it will stay good after that.

Built in proxy validator can also be used as a quick and efficient stand-alone tool to weed out invalid proxies. For example, if you purchased a large amount of proxies and need to find out which ones are definitely not good to return them for replacement. Here is the quick way to do that:

  1. Import the proxies you got from your supplier, this article explains how to do that step by step. You can also add the proxies manually, one by one, but importing is much faster and easier.
  2. Wait few minutes for the proxies to get validated. By default, the software uses 50 threads for validation, and it takes about 1 minute to validate 1 proxy. So If you have 500 proxies, it should take [500 proxies] / [50 threads] * [1 minute] = 10 minutes. 50 threads is a very conservative setting to keep your CPU and network usage low, because validation is supposed to be a background task, that should not consume many resources. But you can safely increase the number of threads if you just need to validate large amounts of proxies at once, it will definitely decrease the total time needed to validate them. This article explains how to increase the number of threads
  3. If you already have the proxies in the software and you want to test them again - simply select the proxies in the list, right click and choose "Schedule for validation".
  4. When the validation is complete, some proxies will be marked green (means good and working), the others will stay gray (means not working). You can easily separate invalid proxies from valid using one of 2 methods:
    • By grouping the list. Click on "Customize table layout" button on the toolbar in the list of proxies, select "Group records" and in one of the boxes select "Valid" item. You will see that the validated proxies will now appear under the "Passed validation" header, and the rest will be placed under "Invalid or not validated".
    • By sorting the list. Click on "Customize table layout" button on the toolbar in the list of proxies, select "Sort records" and right next to the "Valid" field name select either Ascending or Descending. Invalid proxies will appear either below or above the valid ones.
  5. Now that you have invalid proxies in one place, you can easily export them. All you need to do is select invalid proxies and use Export data wizard to send the list to text file or Excel, that you can later send to your vendor. Don't forget to choose "Selected records" in the "Export filter" box on the last page of the wizard, so that it does not export the whole list, but only the selected items.