Indexing your sites on Google
When you have installed the Google indexer on the server and completed the preparatory work for indexing, all you have to do is add links to the indexing queue. Go to the "Add Urls" section and upload links to the queue using the selected method.
All new links are added to the end of the queue, but you can expand the queue in the options.
Piece-by-piece sending of Google Indexing API requests
By default, the indexer works in the mode of sending requests individually. Automation takes place using the Cron scheduler. A scheduler task is performed every 7 minutes, and one link from the queue is sent from each Google Indexing API key uploaded to the admin panel and activated.
If a 200 response was received after sending the Google Indexing API request, then the link was sent successfully. Such a link is moved from the queue to the log.
If another response is received, for example, 403, the link will remain in the queue. It will be highlighted in red, and next to it will be indicated the key from which the Google Indexing API request was sent.
If another response is received, the link will simply remain in the queue until the next launch.
Thus, if you have added, for example, 35 keys, then 35 links will be sent every 7 minutes.
When sending a Google Indexing API request in the piece-by-piece mode, the Google scanning bot scans the page being sent at the time of sending the request. That is, if the link from the queue has gone into log, then the page has already been scanned!
Batch sending of Google Indexing API requests
In the indexer options, you can switch to batch sending of queries. This must be done if you use more than 40 keys. If you do not do this, the script may run with errors, causing a heavy load on your server and on the Google server.
In batch query mode, the Google Indexing API fully processes one key each time a scheduler job is started. According to the Google Indexing API documentation, there can be no more than 100 links in a single batch request. In practice, it turned out that with such a volume of links in one request, Google Bot does not visit all of them. We have reduced this value to 5 links per request. A total of 40 requests are sent, with 5 links each.
All successful requests (with code 200) from the queue are moved to the log. If the answer is not 200, then the links are marked in the queue with the code 999 and highlighted in red. For batch requests, it is not possible to determine the reason for a separate link why the response is not 200. This can be either a 403 error or any other response code, including an error on the Google server side. In order not to interrupt the indexing process due to temporary errors that cannot be fixed on your part, you can disable error fixing in the options.
When sending batch requests, Google Bot scans pages with a slight delay - from 2-3 seconds to 10 minutes.
Automatic indexing in Google
You can set up automatic site indexing in Google using the additional component Sitemaps Checker. The indexing process is no different from manual, the only advantage is that the indexer will scan your site's sitemap itself and add links to the queue automatically when new pages appear or modified, depending on the selected scanning mode.
The deadline for pages to be included in the Google index
Regardless of the mode of sending Google Indexing API requests (piece requests or batch requests), the sent pages usually end up in the Google index from 2-3 hours to 2-3 days.
