A while ago, we decided that our living room is occupied by too many remotes. While this is a common issue when building a HDTV/BD/ATV/Audio rack, the solution is plain and simple: Get a universal remote. I decided for a Logitech Harmony Ultimate which does have quite some track record in terms of building remotes. There are other manufacturers that build such remotes as well and please feel free to evaluate them. I sticked with Logitech since i already own(ed) some of their stuff and it worked well. Also, their system is widely used and a de-facto standard. I like the idea of a more-or-less simple remote and a “invisible” hub quite a lot.

The good

Not going into details or writing some product review, however the Harmony Ultimate remote and the hub are very well built. The touch screen is okay’ish regarding responsiveness and all buttons have a nice feedback and illumination. The tilt sensor is a very good idea to wake up the remote on movement. Overall i’d say hardware is a B+ since i dislike the idea of the LCD splitting the buttons and the hub is perhaps a bit larger than required. I also got a pair of Harmony Precision IR cables to directly attach them to the devices rather than placing the IR blasters. These guys take care of the rack which is now behind closed doors while the hub manages the HDTV and speaker setup.

The bad

Logitech opted to go a all-online configuration. Generally that’s a good idea to spare fiddling around with USB cables and software installation. The hub connects via WiFi and fetches the current configuration when being told to do so. Same applies to the Android/iOS remote app.

What i really dislike about this choice is, that Logitech requires a browser-plugin to be installed. Hell, the 90ies are over! Technically it does not seem to be more than a Browser-USB bridge for initial setup of the Hub and the remote. Apart from that, Logitech opted for a user configuration frontend built with Silverlight… While arguing on Silverlight/Flash/HTML/Java is quite exhausting, i simply state that i don’t like the implementation. It works without larger glitches, but feels quite slow and clunky.

Some very basic features are missing from the Harmony Ultimate: PIN lock of the remote and multi-user support. So in case you don’t want your kids to use the Harmony to play around, you have to hide it. A simple 4-digit PIN lock should not be rocket-science, eh? Even worse than that, you do have one online account at myharmony.com which is bound to your remote. Meaning that if you don’t want to share the credentials with your husband/wife/kids. On top, you cannot change the accounts mail address and your configuration cannot be exported. I’d really hope that Logitech accepts that there is more than one person within a typical household which wants to configure the remote.

The ugly

Apart from controlling some TV/Audio devices, i use a set of remote power outlets (Intertechno IT-1500) to shut down any standby activity of my TV/Audio rack. In order to do so, it’s mandatory to have a piece of hardware that’s compatible with your Harmony and your RF outlets. Harmony Ultimate does use IR and Bluetooth, no RF. In my case, i opted for the LightManager Pro+ which can handle my outlets and is compatible to most RF outlets offered in EU/Germany. This nice piece of hardware gets configured separately to communicate with your outlets. In the end, it offers 254 slots for RF devices and can assign several commands to each (on/off/toggle/dim) as well as time or even temperature based actions.

Integrating the LightManager to your list of devices and activities is quite straight forward since Logitech already knows the IR codes. But to my surprise, i simply could not do anything afterwards! I could customize my Activity and add a command for the LightManager, but it kept being added to the bottom slot of my activities command list. Of course i’d need it in #1 slot since all subsequent devices rely on power supply. Logitech states, that additional commands to an existing Activity must be added to the bottom of the list and that there is no way to re-order them. Damn!

After some quite friendly but not very productive calls to the Logitech support, i did take some time to work around the issue. As it turned out, there is a way!

Turning on outlets before turning on devices

I assume that LightManager is already configured and your remote outlet takes “L001” as “on” and “L002” as “off”.

First, go to the crappy Silverlight abomination which calls itself “MyHarmony” and login. Chose your remote and select “Devices”. Now select your LightManager and click “Change Device Settings” and go for “Power settings” and chose “I want to keep this device on… turn it off when i press the Off button”. This will help saving a lot of time when switching Activities that rely on the same power outlet. At the next step, tell Myharmony, that you use two different buttons for power on and power off. Finally, Myharmony lets you assign power on and power off actions. There, you assign command “L001” for power on and “L002” for power off. When you now add the LightManager to an activity, you can put your LightManager “power on” to slot #1. The downside is, that you need one LightManager “device” for each outlet you want to power on/off, but Harmony Ultimate can handle 15 so that should work out for post people.

Since spying on citizens’ daily communication has shifted from being a rumour for paranoids to being a proven fact, some people think a lot harder about what they can do to preserve their privacy. With E-Mail still being the number one form of communication at the internet, this is one of the most obvious starting points. Due to the nature of the SMTP protocol, spying cannot be shut-down completely, even with content encryption. However, one aspect that can be worked on, is mail storage. Spying on each mail transferred via SMTP is quite expensive for an attacker while just fetching information from a multi-GB mailbox with years worth of social links and information is very cheap.

When relying on secure physical storage, most hosted mail services are not acceptable. They either make money from user data, closely work together with intelligence agencies, cannot provide an acceptable privacy policy - or simply are too expensive for personal usage. A possible solution is to use a local “at home” mail storage. Sadly, just putting some machine as a IMAP/SMTP host to a residential internet line means a couple of drawbacks. For one, most consumer internet connections will use dynamic IP addresses and force regular reconnections/assignments. Second, most telcos don’t offer proper reverse-resolution of the IP, many SMTP hosts use that to identify spammers as well as blocking whole IP ranges which are known for consumer internet connections.

Dynamic IP addressing could be worked around by using popular DynDNS services. However, a DNS MX entry must not be a CNAME but a “real” address. Some mail providers (e.g. GMX) check for this and deny taking mail from such hosts. The issue regarding reverse-resolution can be tackled by buying some mail re-direction services (e.g. dnsexit, dnsmadeeasy) that will relay your mail, but will set you back at least $25 a year with limited confidence to the “privacy” concern and limitations to mail throughput/features.

For me, the best solution has been using a small virtual server (VPS) as a SMTP relay. This gives full control on mail services, features and security. At home, i use a Synology DS214 for IMAP storage and SMTP submission. This box sends all mail to my VPS which then relays that mail to the destination recipient. The other way around it works the same way: incoming mail for my domain is relayed to the DS214 at home. The VPS costs me about €3 a month and of course allows more than just relaying mail. I’d like to share the configuration details with you.

Network hosts

client
The clients mail application uses “ds214.example.com” as IMAP and SMTP server.

ds214.example.com
This host is located at home and uses a residential ADSL line. It uses a dynamic DNS service to update its dynamic IP for “ds214.example.com”. Ports 25 and 993 are forwarded by the ADSL router to allow access by the client and the mail relay (vps.example.com).

vps.example.com
This host does the mail relaying. It is added as DNS MX entry for “example.com”, it’s static IP address reversely resolves to example.com

mx.otherhost.com
Other mail providers hosts just communicate with “vps.example.com”, not knowing anything about it’s relay functionality.

DS214 configuration

This is Synology specific, but may easily be achieved with other MTAs as well.

Postfix configuration

The MTA is running Postfix and Postgrey at vps.example.com is configured like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
biff = no

append_dot_mydomain = no

readme_directory = no

smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

myhostname = vps.example.com
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = vps.example.com, localhost.example.com, localhost
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = 123.123.123.123 # external IP address

transport_maps = hash:/etc/postfix/transport
relay_domains = hash:/etc/postfix/transport
relay_recipient_maps = hash:/etc/postfix/recipients
unknown_relay_recipient_reject_code = 550
smtpd_client_restrictions = permit_sasl_authenticated, reject_unauth_destination, reject_unlisted_recipient, reject_rbl_client zen.spamhaus.org
smtpd_recipient_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_unauth_destination, check_policy_service inet:127.0.0.1:10023

smtpd_sasl_local_domain = example.com
smtpd_sasl_auth_enable = yes
broken_sasl_auth_clients = yes

Relay domains and target hosts are configured at /etc/postfix/transport. In this case, all mail to “example.com” is relayed to ds214.example.com.

1

example.com      smtp:[ds214.example.com]

Valid recipients for this domain are configured at /etc/postfix/recipients. In this case, only mail to any “example.com” address is allowed to get relayed.

1

@example.com OK

Make sure these files are converted to a binary format.
relayed.

1
2

postmap /etc/postfix/recipients
postmap /etc/postfix/transport

When relaying outgoing mail (from ds214.example.com), SMTP authentication is used to make sure no unprivileged users use this server as mail relay. In this case, i’ve used saslauthd for authentication. Postfix is advised to use saslauthd and accept “PLAIN” password authentication at /etc/postfix/sasl/smtpd.conf.

1
2

pwcheck_method: saslauthd
mech_list: PLAIN LOGIN

Restart Postfix after applying this configuration

1

service postfix restart

saslauthd also needs to be configured to accept PLAIN authentication. As the user/pass database, sasldb should be used. This can be done at /etc/default/saslauthd:

1
2
3

START=yes
MECHANISMS="sasldb"
OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd"

Restart saslauthd and make sure the path specified with the -m option can be accessed. This is required since Postfix is supposed to run in a “chroot” environment.

1
2

service saslauthd restart
dpkg-statoverride --add root sasl 710 /var/spool/postfix/var/run/saslauthd

Now we create a user within sasldb which is used to authenticate a client which is relaying mail through this MTA.

1

saslpasswd2 -u example.com -a smtpauth mailrelay

Make sure the user “mailrelay” and the given password is used at the SMTP client, which is ds214.example.com in this case.

When it comes to load testing a web application, usually JMeter comes up as the go-to solution. It has a huge amount of features, protocol support, is opensource, establishment - and boy it has a GUI! There are some other alternatives like pyLot, locust, Grinder, radview, wapt or LoadUI which are either commercial or not as feature rich and less established.

Lately, some hosted solutions like loader.io, neotys, blitz.io, blazemeter, loadstorm etc. joined the party. These offerings are nice for testing the whole stack from an end-user perspective or running limited tests. The downside is, that these are typically sold by bandwidth or threads (users) which is a good business model but comes unhandy when really stress testing an application over a long period of time or testing within an internal network.

Usability

One of the biggest benefits of JMeter, in many peoples minds, has been its graphical user interface. Well, it’s kinda powerful and it kinda allows easy test creation. People tend to think that a GUI makes everything easier since many concepts are well known.

In my opinion the opposite is true, at least for JMeter. Its GUI is quite bloated, which is natural given its complexity and amount of features, but in the end is some button-input-and-knob representation of an XML structure. Under the hood, JMeter generates massive amounts of XML, based on what tests the user defined at the GUI. By itself, this is not bad. However, it shuts down any serious attempt on manual interaction on an XML level. This becomes very visible when using JMeter tests while working within a team. We use git for source control and due to a lack of good and powerful GUI based tools, primarily work with the command line implementation. Reviewing commits or merging is really a pain when you have to diff and compare on a XML level.

Readability of code is usually determined by the skills of the author, in this case the author really doesn’t do anything wrong - but by focusing on a GUI, readability for all other forms of representation suffers. Even if you manually tame the XML output, JMeter will just overwrite it using machine-readable-only code. XML is very well structured but apart from being structured, there are other requirements to good code. Even easy tasks like replacing some parameter or defining properties becomes a pain using the GUI since you have to browse it and manually spot elements. Either you got almost everything within variables, or you’re doomed.

In my opinion, JMeter is nice to rapidly create one-shot tests that can be shown around for education, validation or audit. For any kind of sustainable agile development or testing, it’s next to useless.

Gatling on the other hand uses Scala to define load tests. Scala is probably not as established as XML but (as programming languages in general) it allows to code much cleaner and use the power of a functional programming language. Being object oriented and relatively up to date, it allows re-using code to avoid redundancy and pretty much everything one would expect from a modern programming language. While coding Scala requires some specific toolchain, the compiled code runs within a standard JVM. This makes it very easy to deploy and leverage existing workflows and tools. If you’re a programmer you probably can start with Scala right away. If you’re not familiar with programming, some hurdle may exist but learning a programming language while creating load tests sounds a good deal, right? There is no need to learn each and every bit of Scala right away, tests usually consist of the same fragments that just need to get put together.

Since Gatling is under very active development and currently transitioning from version 1 to version 2, some API vs. documentation hickups and bugs may happen from time to time. The core developers and community is very responsive and most issues get covered within hours.

Performance

The funny thing about load tests is, that many environments actually test the efficiency and performance of the test runner, rather than the application thats supposed to be tested. In projects i’ve seen people building monstrous phalanxes of test clients to stress-test a single server. Distributed test clients and down-sizing both the servers spec and the test coverage seem to be appropriate measures to get the server under load. Lets check some real-world example.

At work we got a nice Intel Modular Server box dedicated for load testing. This baby sports a chassis with redundant PSU, networking, storage and six identical blades loaded with dual Xeons and lots of memory. In a nutshell, this is a datacenter within 6HE where testing can happen without external influences. Earlier, we used one virtual machine as test director and result-parser, 4 machines running JMeter and one machine running the application (including all infrastructure) we wanted to test. JMeter has been configured to spread the test scenario to all 4 machines, effectively cutting the number of threads by 4 and feed them to the test clients to execute them simultaneously. While this worked well, it really felt odd to have 4 specialised machines hammering on one specialised machine to push it to its limits.

What we see at this screenshot is one machine running about 100 concurrent threads (“virtual users”) occupying 4 CPUs. The server on the other hand is also quite stressed but keep in mind there are 4 test machines with 4 CPUs each running 400 concurrent threads in total. Even these 4 test machines did not manage to create significant load at the server, to find out its diminishing or even tensile point. One major drawback with JMeter is that it uses one system thread per user, handling large numbers of active threads is very expensive on the CPU.

When using Gatling, we easily manage to get 1200 concurrent threads running at just 25% CPU load of one CPU of a single virtual machine. This is about 200x more efficient than JMeter (1/64th of the CPU load while creating 3x the load). The server is also stressed quite well and we’re able to push load testing far beyond its tensile point.

Reporting

When running JMeter using its GUI, reports are fairly nice, real time and the tool offers some help to dig through the results. However, when not being able to use the GUI (e.g. for unattended testing, continuous integration), you get a bunch of .jtl files, either XML or CSV. These can then be put into JMeter again for analysis or get processed using XSLT or tools that understand CSV (yeah… Excel). Thats all good and at this point highly structured data makes a lot of sense. There is a variety of tools that help with graphing, charting and analysing of its data. The downside is that you almost always need some kind of extra tool to make JMeter reports understandable - and you always have to wait until the test run finished. At least to my knowledge, there is no realtime graphing apart of the JMeter GUI.

Gatling also creates machine readable data files, but already provides a really nice report module that generates HTML pages with charts and graphs. In addition, the integrated Graphite feeder allows real-time charting of test runs to tools like Graphite or Grafana. This becomes really powerful for showcases or unattended test runs. Overall i think the built-in reporting of Gatling outperforms JMeter by large, even if JMeter reporting may be more accurate and comprehensive on a scientific level.

As with every discussion, there is no “using the wrong tool” - it just depends on the job that needs to be done and some thinking outside the box. There are a lot of great tools that are not (yet) mainstream but help with every days work and contribute to getting better tests, results and software.