How I transformed a Blogger blog to a Nikola two-blog website

L. Amber O'Hearn — Tue, 08 Jul 2014 16:59:38 GMT

In the last post I described why I wanted to move my blog, and what went into my choices. My goal in exporting my site from Blogger to Nikola was to separate my content into two separate blogs on a single site: one which is the primary blog, and the other which resides in the background. I wanted them to be archived separately. I also wanted there to be a front page that didn't change every time I wrote a new post.

Importing the blog

Nikola has a plugin to import a blog from Blogger, so the first step was to import the blog. To import, I took the following steps:

First, I exported my blog from Blogger, which gave me a file called blog-07-05-2014.xml. Then,

$ sudo pip install nikola
$ nikola plugin -i import_blogger
$ nikola import_blogger -o site blog-07-05-2014.xml # The -o tells nikola what directory to make the site in.

After this, I had some cleanup to do. For one thing, Nikola renamed all the Blogger files (in a sensible way). Blogger has date-derived subdirectories for posts, whereas in Nikola all the posts are kept in one folder, called "posts" So when the new site is up, redirects will be required for anyone with the old link.

The import gave me a file called url_map.csv, which contained all the information necessary to redirect the old links to their new locations, but as far as I could tell, those redirects still had to be encoded into the configuration file to take effect. Since I only had a few, I did this manually. For every line in the url_map, I inserted a tuple into the REDIRECTIONS list in conf.py.

The result at this point was a Nikola blog that contained everything the old blog did.

Making it not a blog

There is a document in the Nikola project describing how to make a non-blog site. The instructions boil down to changing three lines of conf.py.

Posts and Pages

As usual, Nikola distinguishes two different kinds of text-derived files.

Posts are blog files. They are archived, indexed by tag, and ordered by date for display.
Pages are essentially independent.

So to make a blog into a non-blog, you simply manipulate the variable POSTS defined in conf.py. POSTS and PAGES are lists describing where to find posts or pages respectively, where to render them, and what template to use for them. We let the POSTS list be empty, so everything on the site will be a page.

POSTS = []
PAGES = [
        ("pages/*.rst", "", "story.tmpl"),
        ("pages/*.txt", "", "story.tmpl"),
        ]

The two entries for PAGES are here to allow either txt or rst, but the one that comes first is what will be used when you use the new_post command.

So you can create a page called "index.html" by running new_post -p, and giving it the name index.

$ nikola new_post -p

[...]

Creating New Page
-----------------

Title: index

[...]
INFO: new_page: Your page's text is at: pages/index.rst

Since we put it at "", which is the top level of the site, it will be what you see on the "front page".

Unfortunately, this creates a conflict, because when you build the site, the blog part is already wired to make an index.html file in the top-level directory, since that's how blogs normally appear. So you intercept this by adding to conf.py the line:

INDEX_PATH = "blog"

This just makes it so the blog index is now created under the folder blog, instead of the top level, and it no longer conflicts.

It also means that now you can have a regular blog under the subdirectory blog, by putting options back in for POSTS:

POSTS = [("posts/*.rst", "blog", "post.tmpl")]

But this is not enough for us, because we have imported posts from Blogger that were also being found from the POSTS list. There is another entry in POSTS that we need back, that tells Nikola to also collect and render existing html files, such as those we imported. So we need to add:

("posts/*.html", "blog", "post.tmpl")

back into the list.

Two blogs

At this point I had part of what I wanted. I had a front page separate from my blog, and all of my previous blog reachable from the site. But I still wanted to have two blogs, a primary one for themed entries, and a journal for unstructured reflections.

Here are the steps I took to factor the blogs apart:

Made a new directory for the journal posts called "journal", and moved the appropriate files into it.
Added ("journal/.rst", "journal", "story.tmpl") and ("journal/.html", "journal", "story.tmpl") to the PAGES list, so old and new journal entries can be found for rendering.
Updated the conf.py REDIRECTIONS to reflect those

This worked to render them and include them in the site, but the journal articles were not indexed. That meant that if I knew the URL, I could go to the article, but a visitor to the site could never discover them.

To add indexing I had to add to conf.py:

STORY_INDEX = True

But again, this created a conflict with multiple files called "index.html" trying to go in the same folder. So I also had to change the name of the index. I chose "index.htm", so that the server would automatically redirect.

INDEX_FILE = "index.htm"

Finally, I wanted the journal to be findable without knowing the directory name "journal", so I updated the navigation links:

NAVIGATION_LINKS = {
 DEFAULT_LANG: (
     ("/archive.html", "Archive"),
     ("/categories/", "Tags"),
     ("/blog/", "Blog"),
     ("/journal/", "Journal"),
     ("/rss.xml", "RSS feed"),
  ),
 }

As an extra configuration tweak, I set TAG_PAGES_ARE_INDEXES = True, so that when you go to the page for a given tag, it renders the posts themselves, rather than a list.

I would like to do that for my journal index as well, but that feature is not yet general, so if you navigate to "Journal" you will get a list of posts, and unfortunately, since it is a journal, they are named by date. Moreover, their tags aren't collected.

Parting thoughts

All in all, I'm satisfied with the move. I got a lot of help from the Nikola community, and my main requirements are fulfilled.

There are a few remaining troubles.

Comments

I signed up with Disqus, and think I have initiated the process of importing my old comments, which I was reluctant to lose. It takes an unspecified amount of time to complete, so I'm hoping that will take care of itself now, but I'm uncertain.

Orphaned rst

Because of the way I built my Blogger site: writing in reStructuredText, converting to HTML, and uploading, I still have the original, pristine rst files on my local system, but Nikola doesn't use them. It uses the backported HTML from Blogger. In order to inject the old rst files into Nikola, however, would require manually editing them all to include the correct headers and timestamps. This seems like a lot of work, and I'm not willing to do it right now.

Moving my site off of Blogger

L. Amber O'Hearn — Tue, 08 Jul 2014 16:48:18 GMT

I was having two problems with the setup of this website. First, I wanted to factor out the experiment in vulnerability and transparency that I have been doing by keep a log of my daily goals, progress, and insights at Hacker School. I like the experiment, but I wanted it to be separate from articles I write more deliberately.

The second problem was that I wanted to migrate from Blogger.

Blogger has advantages. It is easy to set up, and freely hosted. It has themes, comments, and a variety of plugins. You can export your stuff if you want to, so you aren't completely locked in.

However, Blogger is not a good match for someone who wants fine-grained control over her content. My use of Blogger for the three sites I have hosted on it has consisted of the following elaborate dance:

Edit my post in reStructuredText [1].
Convert my post into HTML using a custom script I adapted from rst2blogger [2].
Cut and paste my post into the Blogger compose form, click Preview, and see if it looked ok.
Repeat until all typos and other issues were resolved.
Click Publish.

The result of all this work was a site that looked more-or-less how I wanted it to in some ways, but was frustrating in others. For example,

I couldn't change the css styles that went with a given theme, (and some of them were really dysfunctional).
I couldn't make the site not a blog — the blog assumption is that your most recent content ought to be your most prominent, and this is not an appropriate assumption for some of my use cases. I sometimes found myself putting off making a post that was less compelling until I knew I could follow it with a better one quickly! The restrictiveness wasn't serving me.

Also, a little independence from Google seems healthy.

What I really wanted was a static site generator, with no dynamic logic on a server-side database. I wanted my whole site complete and rendered on my local machine to do with what I liked. In other words, I wanted a static site generator.

Choosing a static site generator

My criteria were:

Open source
Python
Works to generate non-blog sites.

This gave me two choices that I knew of: Pelican, and Nikola.

I have had my eyes on Pelican for some time now. I didn't like the way the resulting websites looked, though, until recently. They had a jarring "I'm a programmer, not a designer" feel. Being a programmer and not a designer, I can't articulate it much more precisely than that. These days, the sites look fine to me. The docs are well written, the code looks good, and people whose opinions about such things I respect use it.

In the meantime, I also found out about Nikola, which was recommended by another respected coder-friend.

I decided to use Nikola, in part because it has a plugin to import from Blogger, and in part because the above-mentioned friend offered to help me.

In the next post, I'll describe how I ported my Blogger blog into a Nikola blog.