Some time ago I created dokk1nfo.dk which is a bookfinding tool for the largest public library in Scandinavia, called Dokk1. The tool simply gives the user a search field and returns the results, as can be seen below
Clicking any of the found items will generate a map with a marking of the shelf which the item is placed on, as can be seen below
This system has been shared on Dokk1's official facebook page
and they even set up a big demo screen at the main entrance, so people could test it out.
Eventually, I was even interviewed by Aarhus Kommune who made a post about it too
so it really doesn't take much to stir it up.
As the name suggest, this website deals with info about Dokk1 and in this post I will delve into some details of how I made it happen.
The current way
Currently you "locate" a book at dokk1 by going to aakb.dk and search for an item/subject. Then you click on the desired item and first then you are able to see if the book is even available at dokk1. That's a lot of clicking and does not give a clear overview (in my opinion).
While writing my master's thesis I frequently visited dokk1 for more information. I quickly became frustrated by using aakb and wandering around looking for the shelfnumber, often ending up asking a librarian instead. Not really what I expected from the best library in the world. But as with most other public projects, it's more flash than content. Thus, inspired by a previous project navnav.dk, I decided to create my own solution.
Navnav uses static images since we knew that the number of rooms probably would not change and static images are simple and can be presented properly in almost any browser. For Dokk1nfo I decided to go with dynamic plots instead since I wanted to be able to indicate the availability of the book using color, plus the system should be able to adapt to any changes in the book database.
So what I needed was
- A record of coordinates & shelfnumber of all shelves
- A way to plot maps dynamically
- Create a search function
The first couple of steps were "easy", in the sense that I basically had to walk around and write all the shelfnumbers down on a drawing. Later I used some Python to create static background images such that the dynamic part "only" consists of plotting a dot on the relevant shelf/shelves.
The search function turned out to be a real headache (and still is) since the underlying book database has changed its structure multiple times.
Getting the data
As mentioned, aakb.dk, is the current way of locating books at dokk1 so I decided to basically scrape the results from that site.
After you press the search button on dokk1nfo.dk, the search is passed on to aakb.dk behind the scenes and the results are then scraped for a few things:
- Faust number (unique id for each item)
- Thumbnail (if available)
So we are still missing some information about where the item is placed. For that, we need to make yet another call using the newly found unique id, the Faust number. After digging around at aakb.dk I found that you could actually get a nice JSON output by doing this
(Notice how there is a "html" part in their JSON object !!)
Which gives information about
When I first started this project, it was actually possible to get results back for multiple faust numbers at once (using /faust1,faust2, etc.) but there is currently a bug (?) where the returned placement will be the same for all the searched numbers if done so. This means that if the first part of the scraping gives me, say 10 faust numbers, I will have to make 10 calls to the database to get all the correct placements!
Luckily it's possible to do a few calls in parallel at once to lower the loading time, but still.
Gluing all this together the flow is like so
dokk1nfo.dk (client) → dokk1nfo.dk (server) → aakb.dk → dokk1nfo.dk (server, scraping in php) → dokk1nfo.dk (client, plot in JS)
As seen, I chose to write the scraping in php such that it could be done serverside to avoid any browser specific DOM handling. The scraped data is then transferred to the client, where the dokk1nfo bookshelf coordinates are found and maps are drawn dynamically.
And that's it (currently)!
- Solar power
- Number of people in line
- Number of people currently inside Dokk1
The data for all of the above is publically available through opendata.dk
As always, the full code for this project can be found at my github.