Forum Moderators: phranque

Message Too Old, No Replies

https - post move monitoring

         

guggi2000

7:07 am on Mar 24, 2017 (gmt 0)

10+ Year Member Top Contributors Of The Month



I would like to share some of the know-how we gained after the move to https. Most threads are about the preparation, this thread should be about monitoring and the post-move experience.

Here our input:

1. 2 days after the move we saw that Google crawled 80% of our pages in 1 day. This resulted in 2G additional download. Be prepared for this spike or limit the crawl rate in GSC or move in smaller chunks.

2. https appeared quickly in the SERPs. Monitor this manually with various Google country specific TLDs, mobile, etc. In the mobile SERPs we did not see https after 3-4 days. It takes a few days to propagate.

3. GSC stats are not reliable for real time. It is a good tool for long term monitoring though. Errors may appear for a few days, such as "robots.txt not found" or "Total Index=0" even when you have >1000 pages already in the SERPs.

4. 301 redirects: Most https Rewrite Rules you will find on the net will forward everything. In some cases you may consider not forwarding certain files, such as robots.txt or sitemap.xml

5. We did not use Sitemaps (yet). The 301 seems to do the job. We could not get a clear answer of the pros and cons of the sitemap. It seems that sitemaps in GSC are good for monitoring, but if you have a good structure and do a 1-1 move, do you really need them? Your input is most welcome here.

6. Monitor your server logs: Make 2 server logs, 1 for http and 1 for https (i.e. you can define that in VirtualHost in Apache).
a. Check in the http-log if it still returns 200 or 404 instead of 301.
b. Monitor the referral traffic from Google in http logs. This should be 0 when the move is complete.

7. Do not make major changes until 1-2 weeks after the move is complete so that the original http pages will look exactly like the https pages. You cannot determine the end of the move simply by looking at GSC. Use your server log instead. Be patient.

guggi2000

7:40 am on Mar 26, 2017 (gmt 0)

10+ Year Member Top Contributors Of The Month



As an addition to point 6, monitor your server logs after the https move:

If you use Apache, add "p:%p" to your LogFormat (for us it was easier than defining 2 logs, because of log rotation issues). This will add the port number to each line in the access log.

Then run the next commands to monitor and count the hits you get from Google SERPs per day, the first is to your http and the second to https (simply by changing the port number):

grep 'p:80' <Access_log_Path> | grep 'google.*/"' | grep '24/Mar' | wc -l

grep 'p:443' <Access_log_Path> | grep 'google.*/"' | grep '24/Mar' | wc -l

Please note that date format may change and make sure "referrer" is registered in the access.log. Replace <Access_log_Path> with your logfile path