Tag: emacs

Circumventing Amazon Geoblocking the Tech-Way

On 15. April 2020

After moving to a different country and having some days of vacations left, one of my concerns was to get the whole home IT setup working again. Precisely, for me this involves setting up my server again, building the network infrastructure and wireless access points as well as ensuring all my everyday services are up and running. One thing I came across when doing this, is the hassle of geoblocked streaming services which I still do want to use. So while my Netflix subscription still works out-of-the-box on my TV, the Amazon Prime Video service is not working anymore. Hence, in this article I will show you how circumventing the amazon geoblocking features is actually possible with only open-source tools.

I can still sign in to my Amazon account and see all the movies and shows. However, nothing is available to be watched in my country, i.e., in Denmark. After a while I realized, I am not looking at my German Amazon interface but the UK one. No surprise, I can login to the UK Prime Video service. But as my Prime subscription runs on the German branch, I do not have access to the UK movies and shows.

What does the TV do?

So, I had a look at the network traffic of my TV. I noticed that the communication with Amazon always started with a call to the website atv-ext-eu.amazon.com. First, I thought this website uses some kind of geolocating my IP address and then determining to which branch I shall be redirected. During transmission of this address, I thought I can intercept the response and alter it to the German branch. Then, I would have my TV once again talking to the German Prime Video. However, there was no response and apparently the forwarding happens on the server-side.

What then?

Normally, people use proxies or VPN services circumventing these amazon geoblocking problems. Unfortunately, my TV neither comes with a proxy or even a VPN configuration possibility. One solution, thus, would be to configure my router to route every traffic via a German VPN. But, then I would always go through this bottleneck, even for services which do not need it. On the one hand my router does not allow for such a configuration, on the other hand, this is not a viable solution for me. Hence, I built my very own approach to solve this problem.

The setup

I have a small server in my network for different services anyway. The idea is simple: intercept all traffic from my TV and route it through a VPN endpoint in Germany.

Using an ArchLinux server as a Router

First step is to configure the IP address of my home network server as the default gateway of my TV. Then, we need to make sure that the server actually forwards IP packets destined for other machines.

sysctl -w net.ipv4.ip_forward=1

VPN connection

For the VPN connection, I ordered a small virtual private server in Frankfurt, Germany, with unlimited traffic and a decent enough bandwidth. This server and the server in my home network are configured to establish a wireguard connection. While the main setup is as usual I took some modifications. The general setup for a VPN server is well explained on the ArchLinux wiki: https://wiki.archlinux.org/index.php/WireGuard

Server setup

The server will be the machine in Frankfurt.

[Interface]
Address = 10.42.0.1/24
ListenPort = 51871
PrivateKey = PRIVATEKEY_OF_VPN_SERVER

PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

[Peer]
PublicKey = PUBLICKEY_OF_HOME_ROUTER
AllowedIPs = 10.42.0.2/32, 192.168.42.180/32
Endpoint = HOME_NETWORK_IP:51871
PersistentKeepalive = 25

In contrast to the default wireguard VPN setups, I added the post up and down directives to ensure the VPN server actually performs network address translation. Obviously, you need to have iptables up and running for this. A second important point is, that we need to include all IP addresses in AllowedIPs which may redirect traffic through this VPN. As I have different subnets for the VPN itself and my actual home network, I have the wireguard address of my home server (10.42.0.2) and the local IP address of my TV (192.168.42.180).

Afterwards, we can just start the network interface with wg-quick up wg0.

Client setup

The client is the machine in my home network.

[Interface]
Address = 10.42.0.2/24
ListenPort = 51871
PrivateKey = PRIVATEKEY_OF_HOME_ROUTER

[Peer]
PublicKey = PUBLICKEY_OF_VPN_SERVER
AllowedIPs = 0.0.0.0/0, ::/0
Endpoint = VPN_ENDPOINT_IP:51871
PersistentKeepalive = 25

Restricting the VPN to Router Functionality

As I wrote in the beginning, I use the server in my home network for several other services which are in parts also publicly available. These services I do not want to route through the VPN but through my original internet connection as I do have a much bigger bandwidth there.

With iptables, wireguard uses a second routing table to separate its rules from your normal routing. On my system, this second table got named 51820. Wireguard then just creates one rule which captures all traffic not marked with 51820 to go through the new second routing table. At the same time, wireguard itself is able to mark all traffic coming from the remote endpoint with that mark. Hence, it is routed using your default table. To prevent routing every traffic through the VPN, we exchange this mark filter for a better suited one. After starting the wireguard interface with wg-quick up wg0, I remove the general routing rule which enforces all traffic to go through the VPN. Then, we replace it with a rule to only use the second routing table if we see packets coming to us on the plain network interface, i.e., enp3s0 on my system. This is done using the iif (incoming interface) rule.

ip rule del not fwmark 51820 lookup 51820
ip rule add iif enp3s0 lookup 51820

Therefore, all traffic originating at the machine itself will be routed using the default table while only packets which arrive on enp3s0 and are to be forwarded will use the second table.

Finally, now I can have every device in my home network choosing between two different gateways, the Danish one at .1 and the German one via the wireguard VPN on .19. Everything needed is just the change of the gateway on the corresponding device and appending the IP address on the VPN server to the AllowedIPs in the wireguard configuration.

Only Drawback

For now, everything is working fine: Prime Video and YouTube play well over this setup. But, for some unknown reason, I cannot connect to Netflix with this setup. The debugging showed it tries to reach three different servers while only two of these connections are successful. I still need to figure out what is wrong here…

Emacs for Academia and Developers

By peter

On 3. May 2019

In workflow

Choosing a suitable editor can be a hard problem. At least it is for me. For several years now, I am constantly switching back and forth between vim, Atom.io, and Visual Studio Code.
Lastly, I really enjoyed using Atom due to its nice looking interface. However, on my system using my set of plugins, VScode was way faster. Realizing this, I remembered there is another editor many people prefer which I never had a closer look at.
So, in this post I will dive into how I setup emacs to suit my writing needs for academia and coding needs for developers.

For now, let us start with the set of plugins I use, why I use them and how I configured them. I am pretty sure there might be better options for some tasks. If you know one, give me a hint in the comments section so that I can have a look at it.

vim Right, the first thing to do in emacs is, get and activate evil-mode https://github.com/emacs-evil/evil
git For using git in my projects https://magit.vc/
package management The default package manager of emacs may feel cumbersome. I very much like https://github.com/jwiegley/use-package
non-distractive writing To have some of atom’s and vscode’s zen-mode beauty, I use https://github.com/joaotavora/darkroom
managing projects If you are working on multiple projects simultaneously, I like https://www.projectile.mx/en/latest/
file browser Very much like nerdtree for vim is https://github.com/jaypei/emacs-neotree If you like eye-candy like me, also get https://github.com/domtronn/all-the-icons.el
fuzzy commands and search This makes every input in emacs work like the fuzzy atom and vscode command search boxes. Just try it and be amazed https://emacs-helm.github.io/helm/ and https://github.com/bbatsov/helm-projectile if you also use projectile

This list only contains my absolute highlights in my config. There are some more plugins I use which you will notice when we have a closer look at the config itself.
Following, however, I just want to note some of the commands I frequently use inside these plugins to manage my day-to-day tasks. Each plugin has detailed descriptions on how to use it, so check the links provided.

Magit

Important commands I use quite often:

commit: c c
push: P p
pull: F u
status/refresh: g
diff: d d
quit: q

Projectile

Important commands I use quite often:

list projects: p
grep in all files of project: p s g

Configuration

Now, the most interesting part is probably the setup in emacs itself. Together, let’s go through my .emacs file and see, what’s there and why. I am afraid I do not comment every line but will try the best to give an impression on what is done and why.

(require 'package)
(savehist-mode 1)
(add-to-list 'package-archives '("org" . "http://orgmode.org/elpa/"))
(add-to-list 'package-archives '("melpa" . "http://melpa.org/packages/"))
(add-to-list 'package-archives '("melpa-stable" . "http://stable.melpa.org/packages/"))

(add-to-list 'custom-theme-load-path "~/.emacs.d/themes")
(load-theme 'atom-one-light t)

The block above ensures we can load plugins (l. 1, 3-5), the history of commands used is stored across sessions (l. 2). Finally, we load our favorite color/syntax theme atom-one-light.

(setq package-enable-at-startup nil)(setq inhibit-startup-message t)(setq-default left-margin-width 2 right-margin-width 2)(set-window-buffer nil (current-buffer))(global-linum-mode t)(package-initialize)(unless (package-installed-p 'use-package)  (package-refresh-contents)  (package-install 'use-package))(eval-when-compile  (require 'use-package))

Next, we setup the window to start cleanly (l. 2-5) and initialize the package management (l. 1, 6-12).

(use-package evil
  :ensure t
  :init
  (setq evil-want-keybinding nil)
  :config
  (evil-mode t)
  (modify-syntax-entry ?_ "w")
  (use-package evil-commentary
    :ensure t
    :config
    (evil-commentary-mode))
  (use-package evil-leader
    :ensure t
    :config
    (global-evil-leader-mode)
    (evil-leader/set-leader ",")
    (evil-leader/set-key
      "m" 'helm-M-x))
  (use-package evil-collection
    :ensure t
    :config
    (evil-collection-init 'outline)))

The above code initializes the evil plugin as well as some additions to that. Note the special directives of the use-package plugin.
The :ensure t clause installs the necessary plugin if it is not present. This enables the reuse of the .emacs config on a different machine as long as use-package is installed there.
Commands in :init are executed before the packet is actually loaded, commands in :config right after loading.
In the case of evil, I define the word text object to include underscores like it does in vim. Additionally, I define , to be my leader key and map the helm plugin to ,m.

(use-package magit
  :ensure t
  :config
  (evil-leader/set-key
    "g" 'magit))

We use magit and map it to ,g.

(use-package darkroom
  :ensure t)
(use-package markdown-mode
  :ensure t
  :commands (markdown-mode gfm-mode)
  :mode (("README\\.md\\'" . gfm-mode)
         ("\\.md\\'" . markdown-mode)
         ("\\.markdown\\'" . markdown-mode))
  :init (setq markdown-command "pandoc --natbib"))
(use-package graphviz-dot-mode
  :ensure t)
(use-package sublimity
  :ensure t
  :config
  (sublimity-mode 1))

Writing

We use darkroom and the markdown mode. The :mode directive binds the markdown-mode to the provided file extensions. Additionally, we modify the default compile command to our needs, i. e. pandoc --natbib (see also Academic Writing using Pandoc). As I like graphing with graphviz, we install a graphviz plugin.
Sublimity then provides us with smoother scrolling. However, this is not working very well for me. I am still looking for better options.

(use-package powerline  :ensure t  :config  (powerline-center-evil-theme))

Developing

We use the powerline module with the evil theme to not get lost in emacs’ mode universe.

(use-package projectile
  :ensure t
  :config
  (evil-leader/set-key
    "p" 'projectile-command-map)
  (projectile-mode +1))
(use-package better-defaults
  :ensure t)
(use-package helm
  :ensure t
  :config
  (helm-mode 1)
  (global-set-key (kbd "M-x") 'helm-M-x))
(use-package helm-projectile
  :after helm
  :ensure t
  :config
  (helm-projectile-on))

We use projectile for project management and map it to ,p. Additionally, we activate helm and remap the default M-x command to helm-M-x. This is done, to ensure we always run helm with its awesome fuzzy search. Finally, we activate helm’s projectile plugin.

(use-package flycheck
  :ensure t
  :init
  (global-flycheck-mode))
(use-package elpy
  :ensure t
  :config
  (setq elpy-modules (delq 'elpy-module-flymake elpy-modules))
  (add-hook 'elpy-module-hook 'flycheck-mode)
  (elpy-enable)
  :bind(("M-g" . elpy-goto-definition)))

For python development, we install flycheck, a syntax checker with python support and enable it globally. As primary python development mode, I prefer elpy where we need to activate the corresponding flycheck mode and remap one of my favorite commands, i. e. elpy-goto-definition as M-g.

(use-package php-mode
  :ensure t)
(use-package web-mode
  :ensure t
  :config
  (add-to-list 'auto-mode-alist '("\\.phtml\\'" . web-mode))
  (add-to-list 'auto-mode-alist '("\\.tpl\\.php\\'" . web-mode))
  (add-to-list 'auto-mode-alist '("\\.[agj]sp\\'" . web-mode))
  (add-to-list 'auto-mode-alist '("\\.erb\\'" . web-mode))
  (add-to-list 'auto-mode-alist '("\\.mustache\\'" . web-mode))
  (add-to-list 'auto-mode-alist '("\\.djhtml\\'" . web-mode))
  (add-to-list 'auto-mode-alist '("\\.html?\\'" . web-mode))
  (add-to-list 'auto-mode-alist '("\\.php\\'" . web-mode))
  (add-hook 'web-mode-hook
            (lambda ()
              (outline-minor-mode)))
  (add-hook 'php-mode-hook
            (lambda ()
              (outline-minor-mode)))
  (setq web-mode-engines-alist
        '(("php" . "\\.php\\'")))
  (setq web-mode-enable-current-column-highlight t)
  (setq web-mode-enable-current-element-highlight t))
(defun my-web-mode-hook ()
  (set (make-local-variable 'company-backends) '(company-css company-web-html company-yasnippet company-files))
  )
(add-hook 'web-mode-hook 'my-web-mode-hook)
(use-package emmet-mode
  :ensure t
  :config
  (add-hook 'sgml-mode-hook 'emmet-mode)
  (add-hook 'css-mode-hook 'emmet-mode)
  (add-hook 'web-mode-hook 'emmet-mode)
  (setq emmet-move-cursor-between-quotes t))
(add-hook 'web-mode-before-auto-complete-hooks
          '(lambda ()
             (let ((web-mode-cur-language
                    (web-mode-language-at-pos)))
               (if (string= web-mode-cur-language "php")
                   (yas-activate-extra-mode 'php-mode)
                 (yas-deactivate-extra-mode 'php-mode))
               (if (string= web-mode-cur-language "css")
                   (setq emmet-use-css-transform t)
                 (setq emmet-use-css-transform nil)))))

As I am also doing web development for some projects, I included configuration for the web-mode as well as a php-mode. The awesome emmet plugin is also great. However, I do not have much experience with any of them so don’t blame me for errors in that part of the config.

(use-package all-the-icons
  :ensure t)
(use-package neotree
  :ensure t
  :config
  (evil-leader/set-key
    "t" 'neotree-toggle)
  (setq neo-theme (if (display-graphic-p) 'icons 'arrow))
  (setq neo-smart-open t)
  (setq projectile-switch-project-action 'neotree-projectile-action))

(evil-define-key 'normal neotree-mode-map (kbd "TAB") 'neotree-enter)
(evil-define-key 'normal neotree-mode-map (kbd "SPC") 'neotree-quick-look)
(evil-define-key 'normal neotree-mode-map (kbd "q") 'neotree-hide)
(evil-define-key 'normal neotree-mode-map (kbd "RET") 'neotree-enter)
(evil-define-key 'normal neotree-mode-map (kbd "g") 'neotree-refresh)
(evil-define-key 'normal neotree-mode-map (kbd "n") 'neotree-next-line)
(evil-define-key 'normal neotree-mode-map (kbd "p") 'neotree-previous-line)
(evil-define-key 'normal neotree-mode-map (kbd "A") 'neotree-stretch-toggle)
(evil-define-key 'normal neotree-mode-map (kbd "H") 'neotree-hidden-file-toggle)

Finally, we setup neotree to use all-the-icons, map it to ,t and bind it to the projectile project management. This adds custom behavior to open the neotree view always we switch to a different project. I think that is quite handy to get a quick overview, where you just moved to.