By: Cole Francis, Senior Solution Architect at The PSC Group, LLC.
PROBLEM
There’s a familiar theme running around on the Internet right now about certain problems associated with generating SEO-friendly Sitemaps for SPA-based AngularJS web applications. They often have two funamental issues associated with their poor architectural design:
- There’s usually a nasty hashtag (#) or hashbang (#!) buried in the middle of the URL route, which the website ultimately relies upon for parsing purposes in order to construct the real URL route (e.g. https://www.myInheritedWebApp.com/stuff/#/items/2
- Because of the embedded hashtag or hashbang, the URL’s are dynamically constructed and don’t actually point to content without parsing the hashtag (or hashbang) operator first. The underlying problem is that a Sitemap.xml document can’t be auto-generated for SEO indexing.
I realize that some people might be offended by my comment about “poor achitectural design”. I state this loosely, because it’s really just the nature of the beast. Why? Because it’s really easy to get started with AngularJS, and many Software Developers simply start laying down code that’s initially decent, but at some point they start implementing hacks because of added complexity to the original functional requirements. That’s where they begin to get themselves in trouble very creative. 🙂
If you think I’m kidding, then just try Googling the following keywords and you’ll see exactly what I mean: AngularJS, hash, hashbang, SEO, Sitemap, problem.
SOLUTION
So, the first step is to remove the hashtag (#) or the hashbang (#!). I know it sucks, and it’s going to require some work, but let me be clear. Do it! For one, generating the Sitemap will be much easier, because you won’t need to parse on a hashtag (or hashbang) to get the real URL. Secondly, all the remediation work you do will be a reminder the next time you think about taking shortcuts.
Regardless, after correcting the hashtag problem, you still have another issue. Your website is still an AngularJS SPA-based website, which means that all its content is dynamically generated and injected through JavaScript AJAX calls.
Given this, how will you ever be able to generate a Sitemap containing all your content (e.g. products, catalogs, people, etc…)? Even more concerning, how will people find your people or products when searching on Google?
Luckily, the answer is very simple. Here’s a little gem that I recently ran across while trying to generate a Sitemap.xml document on an AngularJS SPA architected website, and it works like a charm: http://botmap.io/
I literally copied the script on the BotMap website to the bottom of my shared\_Layout.cshtml file, just above the closing tag. This gives BotMap permission to crawl your website. After doing this, push your website to Production, then point the BotMap website to your publicly-facing URL, and finally click the button on their website to initiate the crawl. One and done!
BotMap begins to crawl and catalog your website as if it was a real person browsing it. It doesn’t use CURL or xHttp requests to determine what to catalog. The BotMap crawler actually executes the JavaScript, which is how it ultimately learns about all of the content on your website that it will use to construct the Sitemap.
This is why it’s so great for websites created using AngularJS or other JavaScript frameworks where content is injected inside the JavaScript code itself. Congratulations, {{vm.youreDone}}!
Thanks for reading, and keep on coding! 🙂
Möbius Straits 