Like most websites and certainly most blogs, this website is powered by the open source WordPress CMS. WordPress comes out of the box with a lot of great functionality and has a plugin infrastructure to allow the core functionality to be conveniently extended.

While WordPress default settings are generally okay to get a site to show up in Google, most websites use a 3rd party plugin to help optimize the configuration. These optimizations could include configuring meta data on each web page, generating specially formatted files to tell Google what URLs exist on a site, restricting certain URLs from being crawled or which can be indexed and a raft of other features.

The image below is from Google Search Console for https://www.lattimore.id.au and shows how many URLs Google is aware of and how many are actually indexed/discoverable within search. WordPress is generating about 900 URLs in total and in the first half of the chart approximately 150 are indexed. The decline midway through is caused by switching from HTTP to HTTPS recently.

The next image is for https://www.lattimore.id.au (after switching to HTTPS) and importantly after reviewing the configuration within the All In One SEO plugin.

The following changes have been made to help Google crawl/index the site more efficiently:

• Applied noindex to date based archives (/<year>/<month>/<day>/)
• Applied noindex to tag based archives (/tag/<tag-name>/)
• Applied noindex to author based archives (/author/<author-name/)
• Applied noindex to search based URLs (/?s=<query>)
• Removed noindex from paginated category based archives (/category/<cat-name>/)

The broad based noindex changes above have cut the number of generated URLs in half, most of which were thin, low quality or broadly duplicative of other URLs in the site.

The second major change was indexing the paginated category archives, such as /category/search/2/. Before the change the top level category URLs, such as /category/search/ were indexed but pages two and onwards were noindex.

Interestingly, Google had a crawl path to the posts via the Archives link in the main menu and also each of the date based archives that were noindexed. I’m now curious if the 300+ URLs which are currently indexed remain that way in the mid/long term or if it is just short lived fame due to them being ‘new’ URLs from switching to HTTPS and it had nothing to do with deindexing the various archive type URLs above.

Only time will tell ;-)