In this blog entry I’m going to talk about what I have been doing this week. Spoiler alert: not that much :(
Before starting with the post, I’d love to thank everyone that has showed interest / support to the project, it really means a lot. I was really impressed by the “repercussion” last week’s entry had.
That being said, let’s jump on what has been done this week.
The first thing I started with was MusicBrainz integration. I spent quite some time coding a client for the MusicBrainz API. It was then when I decided to drop all the progress I had and just use gomusicbrainz. This decision was taken because my implementation wasn’t that good and there was already someone that did the work for me (though I’m going to need to write some parts).
After “finishing” part, I decided to go with the configuration next, since the MusicBrainz part has some configurations. Marmota is going to be using Viper for managing the configuration. This is because it lets the user decide which syntax wants to have in their configuration file, enables reloading the configuration without having to stop and start again the service and saves a lot of time (which is precious to me).
The next step was the database. I, once again, chose to use a third party library for managing the database. The choice was Gorm. This is because it enables the user to choose which database wants to use, saves a lot of time to the developers, it makes really pleasant to do database operations and it’s what I’m (probably) going to use at work so I’m not going to need to learn another “framework”. Also, I think it’s going to be really useful to have an ORM in the future for specifying all the relationships between the models.
Next, I started to work with the file upload. I knew it was going to be hard, so I splitted the function into 4 pieces:
- Write the file to the filesystem (the server recieves it as a byte array)
- Add the file to IPFS
- Pin the IPFS file (so it doesn’t disappear)
- Add the file entry to the DB
The first and the last steps were easy to do. It was hard to start because I wasn’t really sure how to structure the code but they were finished before me realizing it. Then the problems came. I knew working with the IFPS library wasn’t going to be easy, but I didn’t knew it was going to be that frustrating. The first thing was the lack of documentation. They have a really simple example about how to use it, but there’s nothing more appart from it. Nothing. I spent a lot of time searching for tutorials and searching through the GoDoc page and when I was ready to start, the big problem appeared: GX.
GX is “a packaging tool built around the distributed, content addressed filesystem IPFS. It aims to be flexible, powerful and simple”. IPFS is built with GX, which forces you to use it too. But with the new Go modules, there’s no need to use an external package manager. In fact, it’s a bad idea, since it ties you (and all the other developers) to a project that might die. Also, it’s not welcoming for new contributors to have to setup a really complicated package manager. It seems that they are discussing to drop GX or, at least, support Go modules: #5850.
Then, there are 3 possible paths to follow:
- Wait until it gets solved (this would block Marmota’s development for some weeks / months)
- Use the IPFS HTTP API
- Use IPFS directly using the command line
I’m not sure which option to decide, so you could suggest me which do you think is a better option and why!
I think that testing is almost as important as the code itself. Code needs to work, so testing is crucial. Also, it really helps to possible contributors to see that they haven’t break anything with their changes. That’s why Marmota is going to use two libraries:
Testify is a really popular choice because it enables you to make really simple and clean assertions and write mocks easily.
Afero is a really cool (and useful) package that I wish I had discovered before. It “provides significant improvements over using the os package alone, most notably the ability to create mock and testing filesystems without relying on the disk”. This is really useful since it simplifies the testing a lot.
For the next week I want to archive the following things:
- Finish the upload function
- Add tests to the packages that currently don’t have
- Start with the ActivityPub implementation (thanks @email@example.com for being awesome and having [almost] finished the V1 of go-fed!
- Fix issue #1
With this post might seem that I have done a lot of things this week, but that’s not true. I struggle finding time for Marmota, since I have a part-time job at the mornings and I study ad the evenings. That being said, I’m really excited to see how Marmota is starting to take shape and I’m enjoying a lot the time I’ve spent so far. I’m learning a lot of interesting things (since a lot of the time I’ve spent this week was reading documentations) and this is also helping with the extra motivation I need sometimes!
If you want to follow the development progress, you need to check the develop branch of Marmota and not the master, since I use Git Flow. Also, the upload part hasn’t been commited yet, so you’re aren’t going to be able to check it out!
Call for developers (and non developers)!
But if you don’t know how to code / don’t want to, you can also help! Since now I’ve been using this image as Marmota’s logo, but I’d really love if you could design / draw one! The idea I have in mind is a really cute marmota copule but, of course, it can be anything else!