An overview of how I reimplemented most of the functionality of a `.gitignore` file in a small amount of bash.


What is a .gitignore file   slide

  • A plain text file
  • A simple syntax for specifying files
    • (aka “globbing”)

Some examples:   slide

# Specific filenames

# Directories

# Wildcard Patterns

... some others

To see more check out

Why did I want it?   slide

Good question…

A project of mine called Hermit   slide

Linking Files (selectively)   slide

  • I needed to be able to setup symbolic links
  • But it would be nice to be able to ignore (not link) certain files

How is it implemented?

First, decompose the problem   slide

  • git-ls-files: print out all non-ignored files
  • Okay, how do I implement that?

Notes   note

First, let’s decompose the problem a bit more. Instead of worrying about how to selectively link most files in a directory tree into the corresponding tree under a different root, let’s transform this into something that’s more approachable.

My initial attempt to re-use git’s implementation led me to find out about the command git-ls-files which basically does an ls on your git repository printing out only files that have not been ignored.

This seems like a useful way to break up the problem.

The Implementation   slide

    find . -type f -o -type d -name .git -prune

    find . -type f -name .hignore
	find -X . -type f -name .hignore | xargs -n1 dirname |
	while read dir
	    sed 's|^|'"$dir/"'|' "$dir"/.hignore
    } | xargs -n1 find . -type d -name .git -prune -o -type f -path

} | sort | uniq -u | cut -c3-

Wat?   slide

The more readable version   slide

    # Print the list of all files, but don't recurse into the .git folder
    find . -type f -o -type d -name .git -prune

    # Print out all .hignore files a second time so we ignore them
    find . -type f -name .hignore

    # Concatenate a listing of all .hignore files, with the path to the
    # ignore file it came from prefixed to each pattern
	# Process all directories with a .hignore file
	find -X . -type f -name .hignore | xargs -n1 dirname |
	while read dir
	    # Prefix the contents of each .hignore file with the path
	    # to the file it came from
	    sed 's|^|'"$dir/"'|' "$dir"/.hignore

	# And finally, print out all of the files that match each of the
	# patterns from all .hignore files. Using find and the -path
	# option allows us to respect the relative placement of each
	# pattern in the directory hiearchy.
    } | xargs -n1 find . -type d -name .git -prune -o -type f -path

    # Now, sort and then print only the unique lines.  This works
    # because anything that matched an ignored pattern was printed
    # once by the first find, and then once by the ignore find, thus
    # duplicating it. Also, remove the leading ./
} | sort | uniq -u | cut -c3-

Let’s take that step by step   slide

All the Programs   slide

  • Standard *nix filters
    • sort : Sorts the sequence by some ordering (can be numerical, lexicographic etc.)
    • uniq : Do operations relating to uniqueness
    • cut : Selectively reprint only a portion of each input line
  • find : list files matching specified criteria
  • xargs : build commands to be executed
  • dirname : print all but the last element of the given path
  • read : Read one line from input and assign it to the name given (shell builtin)
Notes   note
  • six programs and one shell builtin
  • find produces output from a command line
  • xargs is more complicated (we’ll talk about it later)
  • sort, uniq, cut operate in “standard unix style” - read from stdin, write to stdout. Also called “filters”

Some interesting syntax   slide

  • Curly braces
  • piping into while loops
    <generate some output> |
    while read <var>
      <some stuff>

Print all the files (except .git/)   slide

# Print the list of all files, but don't recurse into the .git folder
find . -type f -o -type d -name .git -prune

Print out all of the files in the directory tree rooted in the current directory:

find . -type f

But, we ignore the subtree starting in the directory (-type -d) named .git (-name .git).

Notes   note

Ignoring the .git folder is key because it contains many files, none of which we are interested in.

The Strategy   slide

Now, we need to talk about what the general strategy for this is going to be.


This is a dummy header that prevents the last one from being missed as a slide.