Using Keybase pgp keys for github gpg verification

September 28, 2017

I recently started using the excellent keybase.io app for encrypted chat and all the crypto goodness motivated me to finally set up gpg verified commits on Github. I started with this helpful article: Github GPG + Keybase PGP which I recommend you take a look at, but I had to do a few more steps that I wanted to document.

Installing Keybase

Keybase is easy to install: keybase.io/download. For arch, it was a simple install and setup:

packer -S keybase-bin  
run_keybase

Once installed, you’ll need to start up the app and create an account if you haven’t already.

Generate a key

Generate a new key with keybase, and upload it to your profile. Alternatively, use keybase pgp select to use your an existing key. To use this key for Github verified commits, it will need to have the same email as on your Github account.

$ keybase pgp gen

Export your keybase secret key to your gpg keyring:

$ keybase pgp export -s -p | gpg --allow-secret-key-import --import --

List keys in your gpg keyring and locate your keybase key

$ gpg --list-secret-keys --keyid-format LONG
/home/jay/.gnupg/pubring.kbx
----------------------------
sec   rsa4096/C17228D898051A91 2017-01-30 [SC]
      326DA75610069B5DECA8D2DDC17228D898051A91
uid                 [ultimate] Jay Baker 
ssb   rsa4096/7C87801D5E56F673 2017-01-30 [E]

sec   rsa4096/C24CD98AB0900706 2017-09-28 [SC] [expires: 2033-09-24]
      F21FC721B22B0C176BAFBE35C24CD98AB0900706
uid                 [unknown] Jay Baker 
uid                 [unknown] Jay Baker 
ssb   rsa4096/4599729752E8D5C4 2017-09-28 [E] [expires: 2033-09-24]

I have two keys here, the second one is the one I made with keybase pgp gen. We want to grab the keyid string from the second line:

sec   rsa4096/C24CD98AB0900706 2017-09-28 [SC] [expires: 2033-09-24]
C24CD98AB0900706=

Let’s set a trust level for our key. Since we just made it, we can give it full trust.

$ gpg --edit-key C24CD98AB0900706
gpg (GnuPG) 2.2.1; Copyright (C) 2017 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Secret key is available.

sec  rsa4096/C24CD98AB0900706
     created: 2017-09-28  expires: 2033-09-24  usage: SC  
     trust: unknow      validity: unknown
ssb  rsa4096/4599729752E8D5C4
     created: 2017-09-28  expires: 2033-09-24  usage: E   
[unknown] (1). Jay Baker 
[unknown] (2)  Jay Baker 

gpg> trust

Please decide how far you trust this user to correctly verify other users' keys
(by looking at passports, checking fingerprints from different sources, etc.)

  1 = I don't know or won't say
  2 = I do NOT trust
  3 = I trust marginally
  4 = I trust fully
  5 = I trust ultimately
  m = back to the main menu

Your decision? 5
Do you really want to set this key to ultimate trust? (y/N) y

sec  rsa4096/C24CD98AB0900706
     created: 2017-09-28  expires: 2033-09-24  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/4599729752E8D5C4
     created: 2017-09-28  expires: 2033-09-24  usage: E   
[ultimate] (1). Jay Baker 
[ultimate] (2)  Jay Baker 

gpg> quit

Now we can decide to do global or per repository signing with git. This step is optional, you can always manually sign commits with git -S or git --gpg-sign on a per commit basis.

$ git config commit.gpgsign true

This is per repository, add a --global flag after config if you want to enable gpg signing globally for git. If we change our minds later and want to disable signing just run the same command but with false. Also, if want to do just do a single commit without signing:

$ git --no-gpg-sign commit

Now let’s tell git which gpg key to use:

$ git config user.signingkey C24CD98AB0900706 # per repository

Again, add a --global flag if you want.

To verify that our commit worked:

$ git log --show-signature
commit 1f10113fadeae03fd8de870fb18c8563d0b3c602 (HEAD -> master)
gpg: Signature made Thu 28 Sep 2017 17:23:20 EDT
gpg:                using RSA key F21FC721B22B0C176BAFBE35C24CD98AB0900706
gpg: Good signature from "Jay Baker " [ultimate]
gpg:                 aka "Jay Baker " [ultimate]
Author: Jay Baker 
Date:   Thu Sep 28 17:23:20 2017 -0400
	detailed commit message goes here

Add our key to Github

Finally, we need to add our key to Github. Remember, your key will need the same email as your Github user email. You can add more email addresses to your key with gpg --edit-key and then the adduid command.

Let’s get our public key from keybase:

$ keybase pgp export
-----BEGIN PGP PUBLIC KEY BLOCK-----

mQINBFnNUb8BEAC3RGNiW3AYUIxAsrQBRfclM65naI/xGlvRju6b5tuoZ33Qbvnq
...
WB+E6/rYlZG4Vdk2W1bTk0R2iAVHoamZD0PmJAkv46SiuHqeyOdBGAGsgdVo1FGa
Gw==
=sE0m
-----END PGP PUBLIC KEY BLOCK-----

Copy that, and head to https://github.com/settings/keys, click “New GPG key”, paste it in, then click “Add GPG key” to save it. verifi

That’s it! You can now have verified commits on Github with your keybase pgp key!

Tags

Useful Linux Desktop Tools for System Administrators

September 8, 2016

Though the demand for Linux Systems Administrators is growing, the sysadmin world is still dominated by Windows in many ways: company laptops will come pre-installed with Windows, Outlook e-mail servers, Microsoft Office formatted company documents, etc. Most businesses now see the value in replacing windows with Linux on their servers, but stop short of replacing the familiar Windows desktop with Linux. But a growing number of systems administrators (like myself), have made the switch to working solely on the Linux desktop. Let me tell you both why I did, and the awesome open source tools I used to do it.

Why Linux?

First off, let me preface this by saying that not all sysadmins will be able to make the switch. I happen to work for a great company that supports my decision (or at least tolerates it so long as I get my work done), but not everyone will be so lucky. Also, some companies will use proprietary in-house tools that only work on Windows, or they may require more integration with Active Directory than is practical with Linux. Still some others will wish to take the path of least resistance, and focus on making their Windows experience work for them.

For me, Linux isn’t just some OS I throw on a server to deploy a Web app; it’s a way of life. I love free and open source software and everything it has done for the world. Using open source software and reporting bugs is part of the price we pay for that greatness. But aside from that, Linux offers a range of possibilities to increase your productivity.

Most Linux distributions offer the ability to choose from multiple desktop environments, each with their own paradigm and workflow. And each of those DEs offer loads of customization options to further tweak them to your preferences. I have found that one of the best ways to increase your productivity is to get your desktop ‘just right’ so that it works and behaves exactly how you want it to.

Additionally, because open source software is free and Linux distributions have massive repositories of software that can be installed in seconds with a single command; it becomes trivial to try a different software solution if your current app isn’t getting the job done. There’s no hassle of purchasing, licensing, registering, or activating the software: you just install it and use it. You also don’t have to worry about the security risk of executing some random executable you downloaded off the internet, compiled from code no one but that company or programmer has seen. And because Linux distros have centralized package managers, system and software updates can be handled easily, in one place; rather than clicking through dozens of popups and bloated system tray applications that phone home separately to check for updates and report back who knows what kind of usage statistics, or worse …

Ok, enough of my rant against Windows. Let’s get to the software! Most Linux sysadmin software guides I see on the net focus either of monitoring tools for the server, or command line tools. This guide is geared more towards desktop tools. All of the apps listed here have been used by me personally in a professional capacity. This isn’t just some list I threw together from Google, these are the tools I rely on every day to get my work done. I hope this post can help make your switch to Linux as productive and satisfying as mine has been.

Distributions

Fedora

Personally, I use Fedora 24 on my company laptop. The reasoning behind this is I have to maintain a lot of CentOS and Red Hat Enterprise Linux servers. Fedora is a cutting edge distro, dedicated to pushing out the latest packages and pushing the boundaries of technology. The changes and innovations made in Fedora will eventually trickle down to RHEL and CentOS, and I’ll already be familiar with them. With cutting edge distros, there’s always a chance that something will break with an update, but if you’re reading this you’re probably a sysadmin: you should already know about backups … Besides, I’ve used Fedora for almost a year with no issues now. Fedora has an excellent, out of the box ready implementation of Gnome 3 as well, which happens to be my preferred desktop environment when I’m not using i3.

Ubuntu

Though Ubuntu is typically seen as a distro for developers, it would also make a great choice for a sysadmin. It has arguably the largest number of available packages. And with something like 70% of the share of Linux servers in the cloud, most sysadmins will find themselves managing an Ubuntu server at some point. Unlike my Fedora example above, Ubuntu desktop and server are the same underlying OS. I actually struggled quite a bit in deciding between Ubuntu and Fedora for my company laptop. For many sysadmins, Ubuntu (or one of its many flavors) will make the most sense. And while folks like to complain about the Unity desktop, it’s actually pretty good if you ask me. I still prefer Gnome, but Unity can definitely get the job done out of the box with minimal fuss.

Other options

OpenSUSE is doing some great stuff with Leap and Tumbleweed, and is blazing the trail with btrfs. They don’t quite have the package availability of other distros, but their Open Build Service puts other distros to shame.

Arch Linux is hands down my favorite distro. I love The Arch Way and the power and respect that the distribution gives users. It can be daunting for beginners and being a rolling release does mean you will have some breakages from time to time. But it will allow you to build the ultimate, lean, customized distro with everything you want and nothing you don’t. If you’ve never used it before, don’t make your first installation on the company laptop though. It has a steep learning curve …

Desktop Applications

Remmina: The Linux Answer to Remote Desktop Manager

I have to interact with a lot of Windows servers over remote desktop at my job, so remmina is probably one of my most used apps. It uses the freerdp protocol to establish remote desktop connections with Windows machines. Like Remote Desktop Manager, you can save connection profiles for one-click-connecting to remote servers. It also supports VNC, ssh, and other connection protocols. It doesn’t have the same feature set that RDM does, but RDM was bloated, in my opinion. The primary thing that I need Remmina for it does well: saving connection profiles to all of the Windows servers I have to manage. It comes with a handy system tray icon for quick connections. It even supports clipboard sync (probably its most useful and time saving feature).

Remmina is also useful for those times when you need to use a specific Windows application for which there is no Linux alternative. Just remote desktop into a Windows box and do the work there. It takes much less resources than virtualization, and it’s way more stable than using Wine. There have been times when I spent all day using Remmina to RDP into a Windows machine to do some Windows specific work. It’s so stable and responsive, you’ll forget you’re even on a remote desktop.

Syncthing

Syncthing is my dropbox/bittorrent sync replacement. It keeps files synced across multiple devices, through an encrypted peer to peer protocol. While Syncthing is still a new project (not yet to version 1.0), it has come a long with since it’s early days. It has a great GTK client now, an Android app, and even a Windows client. No need to worry about a 3rd party handling sensitive company data, or expensive storage prices … you can sync as much data as you have space for. Everything is encrypted using TLS and perfect forward secrecy. This project has a great future ahead of it and I’m excited to see what new features get added over time.

LibreOffice

LibreOffice is a must have for anyone using Linux as their main desktop. I have yet to encounter an Office document I couldn’t edit in LibreOffice. A lot could be said about LibreOffice, but I’ll leave it here.

Atom or Sublime …

The only things linux nerds love arguing about more than favorite distros is their text editor of choice … well, they also love arguing about systemd, but that’s another post. The argument is usually between vi/vim and emacs. For the record, I’m Team Vim :)

For GUI text editors though, there are a lot of choices on Linux, but the top contenders these days seem to be Atom or Sublime Text. Sublime is feature packed and powerful, but it’s not free as in freedom. You can use it for free, but the license is $70. Sublime is great software, and I’d say it’s worth every bit of that $70 if you use it daily as a sysadmin or developer. But Atom is not only free to use, it’s also licensed under the MIT License … not exactly Stallman’s favorite, but better than proprietary if open source matters to you. Atom has nearly all the features of Sublime, and it has a lot of active development and large community around it.

Mostly, I use a heavily customized vim as my editor of choice. But when I feel the need to use a GUI text editor, I tend to reach for Atom.

So that list was woefully short, and included quite a bit of discussion on distribution selection. There is a whole lot more to cover on the topic of sysadmin tools. I think I’ll do a round up of tiling window managers and command line tools, in my next post on this topic.

Tags

Minimal, Clean Conky

January 26, 2016

Conky is a lightweight system monitor for X. It’s a great way to display all sorts of helpful information right on your desktop. It’s somewhat similar to Rainmeter (for Windows) or GeekTool (for OSX), but it’s far more powerful. Unfortunately, as with many things in Linux, powerful can mean hard to configure. In this guide, I’ll go over my ~/.conkyrc for a minimal, clean conky configuration that provides useful info without polluting your desktop with clutter.

desktop screenshot with conky

My current desktop with conky. I’m on Fedora 23 with Gnome for a desktop environment.

Conky

First, let’s install Conky. It’s available in the official repos of most distributions.

pacman -S conky          #For Arch
apt-get install conky    #For Debian/Ubuntu
dnf install conky        #For Fedora 23, use yum for older versions

Note that configuring Conky with your desktop environment/distribution may require additional packages to be installed. If you’re using Gnome on Fedora 23, then the .conkyrc I provide below will work for sure. If you’re using another desktop environment with Arch, check out the Arch wiki on conky. It provides tons of details on configuring conky, and even lots of little mini guides for how to cusomize your .conkyrc

If your distro doesn’t have a conky package, or if you want to compile it yourself, instructions can be found here.

Once installed, you could start using the default configuration right away. Just start conky with conky &. The default configuration is fine, but didn’t suit my tastes. Conky will look in ~/.conkyrc for any user configurations, so that’s where we’ll put ours.

.conkyrc

Our minimal, clean .conkyrc:

# - Conky settings
update_interval 1
total_run_times 0
net_avg_samples 1
cpu_avg_samples 1
imlib_cache_size 0
double_buffer yes
no_buffers yes

# - Text settings
use_xft yes
xftfont Sans:size=12
override_utf8_locale yes
text_buffer_size 2048

# - Window specifications 
own_window_class Conky
own_window yes
own_window_type normal
own_window_argb_visual yes
own_window_argb_value 255
own_window_transparent yes
own_window_hints undecorated,below,sticky,skip_taskbar,skip_pager
alignment top_right
gap_x 40
gap_y 40
minimum_size 300 550
maximum_width 550
default_bar_size 550 8

# - Graphics settings
draw_shades no
default_color cccccc
color0 white
color1 E07A1F
color2 white


TEXT
Kernel: ${alignr} ${execi 5000 uname -r | sed "s@.fc.*.x86_64@@g" }
Uptime: ${alignr}${uptime}

CPU1: ${cpu cpu1}%${alignr}CPU2: ${cpu cpu2}%
CPU3: ${cpu cpu3}%${alignr}CPU4: ${cpu cpu4}%
Temp: ${alignr}${acpitemp}°C

Memory: ${mem} ${alignr}${membar 8,60}
Disk: ${diskio}${alignr}${diskiograph 8,60 F57900 FCAF3E}
Battery: ${battery_percent BAT0}% ${alignr}${battery_bar 8,60 BAT0}

# Processes
Processes: ${alignr}$processes
Highest: ${alignr 40}CPU${alignr}RAM
${voffset -11.5}${hr 1}
${voffset -4}${top name 1} ${goto 124}${top cpu 1}${alignr }${top mem 1}
${voffset -1}${top name 2} ${goto 124}${top cpu 2}${alignr }${top mem 2}
${voffset -1}${top name 3} ${goto 124}${top cpu 3}${alignr }${top mem 3}
${voffset -1}${top name 4} ${goto 124}${top cpu 4}${alignr }${top mem 4}

${voffset -4}SSID: ${alignr}${wireless_essid wlp3s0}
${voffset -4}Local IP: ${alignr}${addr wlp3s0}
${voffset -4}External IP: ${alignr}${execi 600 curl https://icanhazip.com}

${voffset -4}hal:${alignr}${execi 600 /home/user/bin/pingtest server1.com}
${voffset -4}helper:${alignr}${execi 600 /home/user/bin/pingtest server2.com}

Just open up your favorite text editor and paste in the above. Save the file as ~/.conkyrc. To test it, start conky with conky &. You should now see my conky configuration on your desktop!

A close up view of my conky configuration.

A close up view of my conky configuration.

Conky Config Explained

The first few sections deal with Conky’s appearance and position on the screen. If you want to change colors or positioning, poke around in here. Otherwise, let’s move on to TEXT section, this is the stuff that actually gets displayed on the screen.

First, let’s take a look at this line, and break everything down:

Kernel: ${alignr} ${execi 5000 uname -r | sed "s@.fc.*.x86_64@@g" }

On Fedora, uname -r returns this: 4.2.7-300.fc23.x86_64, which is more info than I want. I already know I’m on 64 bit Fedora 23. It’s just the kernel version that I want displayed. So we use sed to replace fc.*x86_84 with nothing, so we’re just left with the kernel version that precedes it. The * here is a regular expression meaning, any number of any characters. That way, when I upgrade to Fedora 24, I won’t have to change my conkyrc.

Of course, conky has a built in way to display the kernel version, with ${kernel}. I could just use this instead:

Kernel: ${alignr} ${kernel}

But just like uname -r, it would display the Fedora version and whether we have a 32 or 64 bit kernel: 4.2.7-300.fc23.x86_64. If you’re happy with what ${kernel} displays for your distribution, then just leave it. If you want to display just the kernel version, then modify the sed command as needed. sed is a very powerful tool, but it can be a little daunting at first. If you’re new to it, then check out the wikipedia entry for more info.

Let’s turn our attention to the networking section, as you may need to make some changes here:

${voffset -4}SSID: ${alignr}${wireless_essid wlp3s0}
${voffset -4}Local IP: ${alignr}${addr wlp3s0}
${voffset -4}External IP: ${alignr}${execi 600 curl https://icanhazip.com}

${wireless_essid wlp3s0} will display the SSID (wireless network name) of whatever network the wlp3s0 interface is connected to. If your wireless interface is named differently, you will need to change this value. To find your wireless interface name, simply issue ip addr:

It will return information on all of your network interfaces, but we’re just looking for your wireless interace:

2: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
...

To get the local IP address, we use ${addr wlp3s0}. Again replace wlp3s0 with whatever your interface name is. Note that if you have a wired connection, it will not be displayed here. You’ll have to use ip addr to find the name of your wired interface and put it here.

There are several ways to get your external IP address, but I find that the simplest is to do curl https://icanhazip.com. Thus, our conky code is ${voffset -4}External IP: ${alignr}${execi 600 curl https://icanhazip.com}.

Finally, we come to the handy little lines that show whether my servers are up.

${voffset -4}hal:${alignr}${execi 600 /home/user/bin/pingtest server1.com}
${voffset -4}helper:${alignr}${execi 600 /home/user/bin/pingtest server2.com}

/home/user/bin/pingtest is a super simple bash script that pings whatever url you give it and returns “Up” if it gets a response and “Down” if it doesn’t. Save this script to your /home/user/bin:

#!/bin/bash

if ping -c 1 -W 2 "$1" > /dev/null; then
echo "Up"
else
echo "Down"
fi

Alternatively, you could put the script anywhere in your $PATH. You can always check your $PATH environment variable with:

$ echo $PATH
/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/user/bin

If /home/user/bin isn’t part of your $PATH environment variable, you can add it by adding this line to your ~/.profile:

export PATH=$PATH:/home/user/bin

If you don’t have any servers to monitor, just remove those lines or comment them out.

And that’s all there is to my conky set up. Conky is a very powerful tool and can be used to display all sorts of information. I like mine simple and clean, but feel free to go crazy. There are tons of really slick conky configs out there.

Tags

Color Bash Prompt (PS1) with git Integration

December 11, 2015

You’ve seen it in screenshots and guides around the net: colorful custom bash prompts with useful info like what branch you’re currently in and if there are any uncommitted changes. Let’s learn how to make our own custom prompt with git integration!

Screenshot from 2015-12-11 18-17-28

git

Obviously, you will need git installed for this to work. But if you didn’t already use git, then you probably wouldn’t be reading this post so … Anyway, in order for us to add git info to our PS1, we’ll need to find the git-prompt.sh script on your system. On Fedora, it’s located at /usr/share/git-core/contrib/completion/git-prompt.sh, but on Arch it’s at /usr/share/git/completion/git-prompt.sh. If it’s not in either location, then just use the locate command to find it:

sudo updatedb && locate git-prompt.sh

~/.bashrc

Bash looks in a few different places for customizations, but the best place to put yours is usually your ~/.bashrc file. Anything in this file will sourced by Bash when you open a new shell or terminal window. The ~/.bashrc is specific to your user. Other user accounts on the system (like root) won’t source your user’s .bashrc. Learning your way around the .bashrc file is an essential skill for any Linux power-user. There is a lot you can do with it, but we’ll just focus on the PS1, aka the bash prompt.

Most distributions have a simple, default PS1, usually something like this:

user@host $ 

Note: The $ usually indicates that this is an unprivileged user, whereas # denotes the root user

That’s a serviceable prompt, but Bash allows you to customize it to your heart’s content. I’m only going to cover my personal PS1 in this post. If you want something tailored just for your needs and want to create your own custom bash prompt, then a good place to start is the Arch wiki: Color Bash Prompt.

If you just want to use my PS1, then comment out your current PS1 (in case you don’t like the new one and want to go back) and paste the following into your ~/.bashrc

# Simple PS1
#PS1='(\u:\h), \W, '

# Change this to the location of git-prompt.sh on your distribution.
source /usr/share/git-core/contrib/completion/git-prompt.sh

# Creates the bash_prompt function
bash_prompt () {

# Checks to see if the current directory is a git repo or not
gitcheck_branch="$(git branch &>/dev/null; if [ $? -eq 0 ]; then echo "yes"; else echo "no"; fi)"

if [ $gitcheck_branch == "yes" ];
then
        # If we are in a git repo, then check to see if there are uncommitted files
        gitcheck_status="$(git status | grep "nothing to commit" > /dev/null 2>&1; if [ $? -eq 0 ]; then echo "clean"; else echo "unclean"; fi)"

	if [ $gitcheck_status == "clean" ];
	then
                # If there are no uncommitted files, then set the color of the git branch name to green
		git_prompt='\[\033[0;32m\]$(__git_ps1)'
	else
                # If there are uncommitted files, set it to red.
		git_prompt='\[\033[0;31m\]$(__git_ps1)'
	fi
else
        # If we're not in a git repo, then display nothing
	git_prompt=""
fi

# This is what actually sets our PS1. The ${git prompt} is what will display our git status.
PS1="\[\e[0;36m\]┌─\[\e[1;37m\][\u@\h]\[\e[0m\]\[\e[0;36m\]─\[\e[0;93m\](\w)${git_prompt} \n\[\e[0;36m\]└─\[\e[1;32m\][\A]\[\e[0m\]\$(if [[ \$? == 0 ]]; then echo \"\[\033[0;32m\]$\"; else echo \"\[\033[0;31m\]$\"; fi)\[\033[00m\] "

}
# PROMPT_COMMAND is run just before the PS1 is printed. We've set it to run our bash_prompt function.
PROMPT_COMMAND=bash_prompt

The key to all of this working is the PROMPT_COMMAND. Bash will execute whatever the PROMPT_COMMAND is set to just before it prints the PS1, in other words, every time you open a new shell, execute any command yourself, or even just hit the enter key. With the above in your .bashrc, Bash will check to see if the present working directory is a git repo or not. If it is, it will check to see if there are uncommitted files. If there are no uncommitted files, it will append the branch name in green after the working directory in our PS1. If there are uncommitted files, it will append the branch name in red. If the working directory isn’t a git repo, then it will print nothing after the working directory in the PS1.

Oh, I almost forgot to mention. There’s another neat little trick in here that I really appreciate:

\$(if [[ \$? == 0 ]]; then echo \"\[\033[0;32m\]$\"; else echo \"\[\033[0;31m\]$\"; fi)\[\033[00m\]

This bit will check the exit code of last executed command. If it’s zero (successful), then the $ will be green. If the command failed, the $ will be red. Can be handy for commands that don’t tell you whether they succeeded or not.

Screenshot from 2015-12-11 18-18-32

There are of course, other ways to implement something like this. A quick google search will turn up plenty of guides for doing it differently. But ultimately, this is the method that works the best for me. It’s also quite versatile. Having PROMPT_COMMAND call a function that sets the PS1 allows you to do all kinds of conditionals on what your PS1 will look like. You could have different colors for different times of the day, A status indicator for whether you’re connected over ssh or through a local terminal emulator (useful if you have the same .bashrc on all of your remote machines), an indicator for whether you’re connected to your vpn or not. Anything you can do with a bash script, you can put in your PS1.

Tags

Arch linux + emby + Kodi + nginx: The Ultimate Media Server

November 30, 2015

I’ve been using Arch Linux as my media server/htpc for several years now and it’s been incredibly reliable. Some people prefer a versioned distribution with an LTS release for something like a media server, but I want the freshest packages, and I don’t want to deal with the headache of upgrading/reinstalling when an LTS release outlives its usefulness or won’t allow me to get the new packages I want. Recently, I decided to give emby a try, after hearing about it on one of my favorite podcasts, the Linux Action Show.

This guide assumes you have a working Arch Linux installation. If you’re starting completely from scratch, then consult the excellent Arch Beginner’s Guide. And if you don’t have time to go through a full Arch installation (thought I highly recommend it as you’ll learn a lot about Linux in the process), then you can always just go with Antergos, which is basically Arch with a nice installer and some sane defaults.

emby

emby is a media server that can manage and stream your movies, tv shows, music, home videos to a plethora of devices. It’s a lot like Plex, but it comes with all of the best features free to use. It works with a server/client set up. You install and configure emby server, in our case on an Arch Linux rig (though they have support for many distributions and operating systems), and then access it from a client like your Web browser, a DLNA client like a PlayStation 4, Kodi media center, the Android or iOS apps, or any number of other options. Check out their download page to see what I mean. Their cross-platform support is some of the best for any app I’ve seen in a while, let alone a media server.

emby

Installing emby

First, let’s create a user account for emby:

sudo useradd -r -s /bin/false emby

emby is available in the community repos. Install it with:

sudo pacman -S emby-server

Now that it’s installed, we’ll need to start the emby service with systemd:

sudo systemctl start emby-server
sudo systemctl enable emby-server

Check to make sure the emby-server service started properly:

sudo systemctl status emby-server
● emby-server.service - Emby brings together your videos, music, photos, and live television.
   Loaded: loaded (/usr/lib/systemd/system/emby-server.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2015-11-29 15:42:16 EST; 23h ago
 Main PID: 15379 (mono)
   CGroup: /system.slice/emby-server.service
           └─15379 /usr/bin/mono /usr/lib/emby-server/MediaBrowser.Server.Mono.exe -programdata /var/lib/emby -ffmpeg /usr/bin/ffmpeg -ffprobe /usr/bin/ffprobe

You should see something like that. If you get errors, check the logs with sudo journalctl -xe. When I first tried to start the service, I got an error saying that emby didn’t have permission to write to the /var/lib/emby/logs directory. If you get an error like that, you can fix it with sudo chown -R emby:emby /var/lib/emby

Once we’ve got the emby-server service started, let’s continue with the set up by pointing your web browser to http://localhost:8096. If you did everything correctly, you’ll see the emby welcome screen. The emby set-up wizard is really very easy to use, so I won’t go over every step, but you basically create a user account and tell emby where your media folders are and what kind of media they contain.

NOTE: emby can only scrape and recognize your media if it is named properly. It supports a number of naming conventions, so you’ll just have to pick one and make sure your media conforms to it. My media was already named properly, so I didn’t have to worry about this step, but there are a few tools, like filebot that can help you get your files in order.

Once you get through the initial set up, take a while to look through the options in the ‘Manager Server’ section:

Emby Manage Server Screen

There are a lot of options you can configure here. Familiarize yourself with all of the options and decide which are best for your set up. You can choose what libraries each user has access to, the max bitrate each user can stream at (useful if you live under the tyrannical rule of Comcast’s data caps), and where to save metadata files (fanart, posters, etc.).

Right now, you have a working emby server set up, you could just open port 8096 in your firewall and access your server from the web with https://your.external.ip.address:8096. emby even has built in ssl (https) support. But I already have nginx web server set up with ssl, and I don’t want the additional security risk and hassle of opening another port in my router.

nginx

nginx (pronounced “engine X”) is an awesome web server that’s fast, configurable and reliable. The configuration can be a little tricky to new users and users coming from Apache, but I actually prefer the nginx syntax now that I’m used to it. In any case, you can just cut and paste my configuration below to simplify the process.

Basically, nginx is going to act as a go-between for clients wanting to access it (and other web services we might be running, like transmission, subsonic, or owncloud). That means, we really only need to have 3 ports open in our router/firewall: http (80), https (443), and ssh (22). It also means, we can set up custom sub-domains, like emby.your-domain.com.

Installing and Configuring Nginx

First, install nginx with:

sudo pacman -S nginx

Once it’s installed, we’ll need to create a configuration file for our emby reverse proxy. Create /etc/nginx/conf.d/emby.conf in your text editor of choice and paste in my configuration below, making sure to change the server name to your own:

server {
    listen 80;
    return 301 https://$host$request_uri;
}

server {
        server_name emby.yourdomain.com; #change this to your domain!
        listen 443 ssl spdy;

        ssl_certificate           /etc/nginx/certs/emby.crt;
    	ssl_certificate_key       /etc/nginx/certs/emby.key;
	ssl_prefer_server_ciphers       On;
        ssl_protocols                   TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers                     'AES256+EECDH:AES256+EDH:!aNULL';
        resolver                        8.8.8.8 8.8.4.4 valid=300s;
        resolver_timeout                5s;
        ssl_stapling_verify             on;
        keepalive_timeout               180;
        add_header                    Strict-Transport-Security max-age=31536000;
        client_max_body_size 1024M;

        location / {
                # Send traffic to the backend
                proxy_pass http://127.0.0.1:8096;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-for $proxy_add_x_forwarded_for;
                proxy_set_header Host $host;
                proxy_set_header X-Forwarded-Proto $remote_addr;
                proxy_set_header X-Forwarded-Protocol $scheme;
                proxy_redirect off;

                # Send websocket data to the backend as well
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection "upgrade";
        }
}

There’s a lot going on there, but this configuration takes any http requests to emby.yourdomain.com and redirects them to use https. It then creates a proxy so that any requests to emby.yourdomain.com/ get redirected to http://127.0.0.1:8096. That’s how we can keep the 8096 port closed. The emby clients will connect to nginx, which will then talk to the local emby server at 127.0.0.1:8096. Any data you send to or from your emby server will be encrypted and secured with ssl.

ssl

Our nginx proxy isn’t ready to go yet, we still need to create some self-signed certificates. Now, you could of course purchase an ssl certificate, but that’s sort of overkill for a home media server if you ask me … But if you already have some certs to use, put them in /etc/nginx/certs, Otherwise, let’s make some now.

sudo mkdir /etc/nginx/certs/
cd /etc/nginx/certs/
sudo openssl req -new -x509 -nodes -newkey rsa:4096 -keyout emby.key -out emby.crt -days 1095
sudo chmod 400 emby.*

The -days 1095 will set how long the certificate will be valid, adjust it to suit your needs. If you don’t understand the other settings, don’t mess with them. Running the openssl command will prompt you for some info, you can leave these blank if you want or fill them in, doesn’t really matter. chmod 400 emby.* will set some secure permissions on the certs so that other user accounts can’t mess with them.

Testing our nginx configuration

Once you’ve completed the above steps, let’s test our new set up.

sudo systemctl start nginx
sudo systemctl enable nginx

With the nginx service started, point your browser to emby.yourdomain.com. You should be redirected to https://emby.yourdomain.com and be presented with the emby server log in screen. Your browser may give you a warning about an untrusted certificate, but that’s just because we’re using a self-signed certificate. We just created the damn things so we know it’s safe. Add a security exception for the cert in Firefox or select Proceed anyway in Chrome. Either way, you shouldn’t be bothered with this message again (unless the certificate is changed).

emby login screen

Success! You’ve now got your own personal Netflix. You’ve got access to all of your media wherever you have an internet connection in a secure, encrypted way. You can make user accounts for your friends and family to use and give them an easy to remember url.

But what about in your home theater? emby’s web client is pretty slick, but no one wants to open up a web browser on their tv to view their local media files. That’s where kodi comes in.

kodi

Kodi (formerly known as xbmc) is open source home theater software. It provides a stylish, themeable, easy to use UI to display and organize all of your media. It has great remote control apps for Android and iOS and supports IR devices as well. If you’re serious about creating the ultimate home theater/media server set up, then at the very least you should look into it.

Kodi can play our local media just fine without emby, and in fact that was the set up I had for years. It works great in this regard and it can handle just about any file type you can throw at it. Guests were always amazed by my kodi set up and how I controlled it with my smart phone (using the excellent yatse app). Where kodi falls short is that it while it can do streaming over DLNA, it doesn’t do transcoding. There’s also not a simple way to keep track of your played position/status if you have multiple kodi instances throughout the house. emby can solve these problems in an elegant way by acting as the backend for Kodi, while letting us keep the beautiful and functional Kodi UI for our big screen.

NOTE: The emby add-on for Kodi will replace your existing Kodi metadata database completely. If you have a working Kodi install, then be sure to make a back up of any user data before proceeding. You have been warned.

That being said, I have found emby to be much better at scraping titles and downloading all of the proper meta images (posters, fan art, banners, logos, etc.) Even though I spent years getting my Kodi database just right, a few minutes of work with emby and I’m happy to throw all that away. I’ve still got my backup of course, but I doubt I’ll ever use it. emby really is that good.

Install Kodi with:

sudo pacman -S kodi

As always, the Arch wiki has a ton of useful info on this package. Be sure to read up on the entry for Kodi if you’re unfamiliar with it.

Installing the emby add-on for Kodi

emby provides their own Kodi repository for their add-on, so installing it is just as simple as installing any other Kodi add-on. Download their repository .zip file.

wget http://www.mb3admin.com/downloads/addons/xbmb3c/kodi-repo/repository.emby.kodi-1.0.2.zip

Then open up Kodi and navigate to System > Settings > Add-ons > Install from zip file.

kodi system settings

kodi install from zip

Once the repo is installed, you’ll need to install the emby add-on itself. In Kodi, navigate to Add-ons > Video add-ons > Emby

install emby add on

Launch the add-on by going to Video add-ons > Emby. If your emby-server service is running, then it should auto detect emby-server for you. Just log in and emby will perform a metadata sync with emby-server. Again, this will completely replace any existing metadata that Kodi has collected. There are a few options you can set in the emby add-on, but if you’re running Kodi on the same machine that’s running the emby server the defaults should work for you.

The emby website has some great screen shots of the emby add-on being used with different Kodi skins to give you a taste of the awesomeness to come:

emby kodi screenshots

The emby add-on for Kodi doesn’t yet support streaming over the web, so if you want to run Kodi with the emby add-on set up on a different machine, you’ll have to share the media over the network using samba or nfs and then set up path substitution in emby, so that kodi can find the files on the network to play. You’ll still be able to track played position/status and see all of your metadata, etc., it just requires some more set up. I haven’t had to set that up yet, so I’ll save that for another guide.

In any case, you should now have a bad ass media center/server set up on Arch Linux!

Tags