Mercurial > ~samis > hgweb.cgi > blag

annotate posts/Managing_dotfiles_with_vcsh_and_mr.md @ 8:03db14198fb3

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
creating tag page tags/tech
author Samuel Hodgkins <samuel.hodgkins@sky.com>
date Fri, 01 Sep 2017 01:16:10 +0000
parents 45362e07941c
children 2307e281b4b7
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
1 Over time, a Linux user may customize and configure his environment rather substantially.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
2 These modifications are stored in a collection of configuration files / data known as 'dotfiles' (because the first letter of many of them is '.').
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
3 For multiple reasons, it is very beneificial if you track, control and synchronise all of your personal dotfiles, a few example reasons include:
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
4 - Having an additional backup
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
5 - Being able to see their history, how they changed over time
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
6 - Being able to rollback changes if needed (I haven't needed this yet)
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
7 - Being able to use the same set of files accross multiple physical/virtual machines
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
8 - Being able to share you configuration with the world so people can learn from it just like you learn from other people's.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
9
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
10 However, there is not one single universal method for managing them, instead there are many tools and approaches that one can take.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
11 GitHub provides a decent list of programs [here](https://dotfiles.github.io/) but I intend to summarize the main approaches below.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
12 (It may be worth noting that while the methods may not be mutually exclusive, there is one 'main' approach/method per tool and that is what counts.)
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
13
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
14 1. Symlink-driven management involves moving the dotfiles away from their original location, instead creating symbolic links to one or more destinations.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
15 There are many ways/approaches of doing this, but the simplest is to just have a single directory be the destination for all the links.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
16 2. VC(Version Control)-driven management involves less management of actual dotfiles compared to the other two. Instead of copying or using symbolic links,
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
17 instead a version-control system is primarily used to track/manage dotfiles in groups. The original dotfiles are left in place, instead they can be treated
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
18 just like every other repository. There are multiple methods of implementing this approach with their own unique advantages and drawbacks.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
19 3. Configuration-driven management involves using explicit configuration file(s) to determine exactly what dotfiles are to be managed/tracked as well as how they are to be tracked among other things.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
20 The key difference between this method and the others is that rather than using interactive commands to manage and modify dotfiles, one or more configuration files are used.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
21 Typical formats for this information include YAML/JSON or a purpose-built configuration format. Typically but not exclusively uses symbolic links for dotfiles.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
22
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
23 I have been tracking my dotfiles for short-to-moderate period of time. I originally started when I read an article about using GNU Stow as the management tool.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
24 Stow has some features that make it just as useful for this as a dedicated too: It supports 'packages' so you can choose to install part of the dotfiles.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
25 It also doesn't make you specify specifically which files to symlink, it just symlinks the entire package.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
26 However, it's definitely not perfect: Symlinks can be overwritten, Moving dotfiles and replicating directory structures sucked, and you could only manage operations from the right directory.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
27 (I could also only easily have 1 VCS repo, which effectively meant private dotfiles couldn't be tracked)
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
28
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
29 One day, while inspecting my ~/dotfiles I noticed that the .git directory was missing. I could've seen this as a disaster, but I didn't.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
30 I had been thinking about migrating away from Stow for a while, but I never actually did anything - so I took this opportunity.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
31 After some reading/googling, I had made the decision to use `mr` and `vcsh`.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
32 `vcsh` would provide each individual repository, public and private while `mr` would be used for higher-level tasks.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
33 There are multiple guides to this pair of tools, such as:
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
34
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
35 * [This very short post on vcsh & mr](https://sumancluster.wordpress.com/2015/05/29/managing-dotfiles-using-vcsh-and-mr/)
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
36 * [This one which links to a more in-depth tutorial but also takes a look at the internals](https://www.kunxi.org/blog/2014/02/manage-dotfiles-using-vcsh-and-mr/)
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
37 * [This very useful and detailed one](http://srijanshetty.in/technical/vcsh-mr-dotfiles-nirvana/)
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
38
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
39 When I was migrating, I particularly found the latter link to be rather useful due to the detailed explanations of multiple common tasks.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
40 However, should you not want to read any of the above links I will attempt to give an overview of how it works in practice.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
41
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
42 # Creating a new repository
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
43
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
44 1. Clone/Initialize the local vcsh repository
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
45 2. Update the myrepos(mr) configuration to include that repository
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
46 3. Add the wanted stuff to the vcsh repository
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
47 4. Write/generate a .gitignore and modify as needed
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
48 5. Commit to the vcsh repository and push both sets of changes as needed.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
49
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
50 # Updating an existing repository
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
51
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
52 1. You can prefix git operations with vcsh and then the repo name to perform them on the repository.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
53 2. Alternatively, use 'vcsh enter' to go into an environment where git can be used normally.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
54
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
55 # Updating *all* the repositories
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
56
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
57 1. Use `mr up` and let myrepos do the job it was designed to do.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
58
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
59 # Bootstrapping the dotfiles
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
60 (presuming git is installed. If not, install it.)
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
61
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
62 1. Install myrepos and vcsh. If there's no distribution package, a manual install is easy (they're just standalone scripts)
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
63 2. Obtain your myrepos configuration.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
64 3. Use `mr up` and let myrepos obtain all your repositories as needed.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
65
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
66 If you think the above workflow looks interesting, I recommend you have a nice read of the above links - especially the last one
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
67 as I found it very useful. However, I have not moved my entire collection of dotfiles over and yet I still have some interesting problems/caveats to tackle.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
68
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
69 Firstly, while using a (private) Git repository to track my SSH/GPG data is useful, certain files have special filesystem permissions which Git does not preserve. While this can be solved with a chmod or two, it may grow
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
70 to be more difficult if I need more of these files in the future - plus I might be able to automate it using mr's 'fixups' functionality.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
71
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
72 Secondly, this is more of an observation than a problem: I'm currently using an Apache-style configuration involving both *'available.d'* and *'config.d'*. This works and is flexible, but it'd be simpler to only have a single directory and have equivalent information be part of the configuration itself.
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
73
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
74 Thirdly, bootstrapping from a completely clean slate is rather complicated. Certain repositories may depend on others to work / be in the correct location. Then there's the problem of access to private repositories, perhaps HTTP(s) could be used to download SSH keys using pre-entered cached credentials? A similar but lesser problem exists with GPG. Public repositories have no issues with this - if need be, they can have the master remote be changed afterwards.s
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
75
45362e07941c Add more old posts.
Samuel Hodgkins <samuel.hodgkins@sky.com>
parents:
diff changeset
76 Anyway, that's all for now. If and when I solve the above issues I'll make sure to explain and blog about each my solutions. Until then, I don't expect this to come up again.