Quick Setup for Hashicorp Vault with Consul Backend

Quickstart to get a Vault instance using Consul as a backing store and be able to manipulate vaults.

Download consul binary and run:

./consul agent -server -bootstrap-expect 1 -data-dir /tmp/consul./va

On the same server, open a new session and download the Vault binary and you’ll also need to create a file called “example.hcl” and populate it with:

backend "consul" {
}

listener "tcp" {
  address = "127.0.0.1:8200"
  tls_disable = 1
}

Start the vault server and you should see:

./vault server -config=example.hcl
==> Vault server configuration:

         Log Level: info
             Mlock: supported: true, enabled: true
           Backend: consul (HA available)
 Advertise Address: http://127.0.0.1:8200
        Listener 1: tcp (addr: "127.0.0.1:8200", tls: "disabled")

==> Vault server started! Log data will stream in below:

[INFO] core: security barrier initialized
[INFO] core: post-unseal setup starting
[INFO] rollback: starting rollback manager
[INFO] core: post-unseal setup complete
[INFO] core: root token generated
[INFO] core: pre-seal teardown starting
[INFO] rollback: stopping rollback manager
[INFO] core: pre-seal teardown complete
[INFO] core: vault is unsealed
[INFO] core: entering standby mode
[INFO] core: acquired lock, enabling active operation

Next, open a new terminal and set an environment variable:

export VAULT_ADDR='http://127.0.0.1:8200'

Proceed with creation:

./vault init

This will output 5 keys and a root token. Proceed by running this 3 times, providing 1/5 keys just generated each time:

./vault unseal

Once complete, run ./vault status and you should see the vault is unsealed.

You can now auth with your root token:

./vault auth 

Lastly, you can now create mounts and save/retrieve secrets and generate tokens etc.

If you see any of the following, you haven’t provided your auth token properly or unsealed your vault:

./vault token-create
Error creating token: Error making API request.

URL: POST http://127.0.0.1:8200/v1/auth/token/create
Code: 400. Errors:

* missing client token
./vault mount aws
Mount error: Error making API request.

URL: POST http://127.0.0.1:8200/v1/sys/mounts/aws
Code: 500. Errors:

* invalid request

Read More

DRBD Device is Held Open by Someone

On a single dual primary setup, the intention was to fail over and this involved demoting primary to secondary and then other node second to primary. This instance in particular was used for nfs. An lsof showing nothing attached to mountpoint and remounting and reunmounting did not fix error.

This failed:

# drbdadm secondary node0
1: State change failed: (-12) Device is held open by someone
Command 'drbdsetup 1 secondary' terminated with exit code 11

Beforehand, a lazy unmount had occurred, which is probably responsible. NFS had not cleaned up properly and somehow still had a process using the mountpoint. Killing nfsd and then re-running the demotion to secondary command was sufficient to finally get the node to secondary.

Read More

Graylog Email Minimal Configuration

Graylog(Graylog2) requires some minimal arguments if you’re wanting to send email from your Graylog instance. This is under the assumption you quickly install Postfix etc and want to test mail.

Even if you’re sending to the local MTA, Graylog will require some inputs. Example working config:

# Email transport
transport_email_enabled = true
transport_email_hostname = 127.0.0.1
transport_email_port = 25
transport_email_use_auth = false
transport_email_use_tls = false
transport_email_use_ssl = false
transport_email_auth_username =
transport_email_auth_password =
transport_email_subject_prefix = [graylog]
transport_email_from_email = [email protected]

This should avoid you seeing the following if you have enabled email but not populated additional required fields:

An error occurred while trying to send an email! 3 days ago

The Graylog server encountered an error while trying to send an email. This is the detailed error message: org.apache.commons.mail.EmailException: javax.mail.internet.AddressException: Illegal address in string ``'' (javax.mail.internet.AddressException: Illegal address in string ``'')

Read More

Dropbox is Making 2FA Pointless

So we know that Dropbox have the power to view your files on the server-side with encryption they can circumvent. They can share your files with law enforcement if required. They can scan hashes of your files and compare them against particular unlawful content they’re targeting. In the past, exploits have been released that allowed anyone to view anyone’s private files. But whatever, let’s say you still want to make use of the Dropbox service but you at least want your private files to remain private.

Let’s say you turn on auto-upload of photos from your mobile device (or even just add photos to your account manually at some stage). And you have a password on your Dropbox account to protect its contents. The intention here is that eventually, a photo you take on your phone will exist on your phone (something you possess; folder not exposed to internet), it will exist on local storage at home (folder not exposed to internet) if you have a dropbox folder set to sync and it will exist on Dropbox’ servers that can be accessed over the web after you authenticate.

Let’s assume that no one has stolen your phone or home computer or that both are encrypted. Now if a hacker wants to get access to your files, they can either hack your account via gaining access to your Dropbox, or gaining access to your email account/password. In the event your Dropbox account password is stolen, hacker has access to your files. In the event your email password is stolen, hacker can reset your Dropbox password there and hacker can access to your files.

Now if you enable 2FA, 2FA is supposed to prevent this happening. If either your Dropbox password or email password is stolen, the hacker will be unable to access your files, even with the right password. They need to also steal your phone and gain access to it to make this possible (encryption makes this tough), and access it before you’ve been able to reset your Dropbox password or disable 2FA from the stolen phone.

The problem is that Dropbox is now embedding your photos in emails they send to you. There are at least 3 of my images in the email I’ve received and the more automatic emails I receive, the more they will embed. The issue is that my photos are now exposed to my email account. If my email is hacked, the attacker now has access to some of my Dropbox content without having to even access my Dropbox account.

So.. why does 2FA even exist on Dropbox? What other crazy ideas are they scheming? Auto-post to your Facebook wall at some random point in the future without warning you?

Pictured below: a section of the email I received, with 3 photos embedded below.

Dropbox

Read More

Quickstart Config of Graylog Centralised Logging

Graylog is a centralised logging service that allows for data analysis. It accepts many data types (GELF HTTP, raw TCP/UDP, syslog content etc) and has a lot of pieces that fit together to enable log analysis and manipulation. As such, the focus of this article is to get an instance up quickly and populate it with minimal data. We’ll use a Docker container that comes comes with Graylog (so elastic search et al doesn’t need to be configured manually) and we’ll populate it with some fake data.

Acquire a VM and install Docker. If you happen to be on something like Ubuntu 14.04 you can proceed with the following command, otherwise look up dependencies:

wget -qO- https://get.docker.com/ | sh

Now we’ll clone the repo for the container and start an instance:

git clone https://github.com/Graylog2/graylog2-images.git
docker pull graylog2/allinone
docker run -t -p 9000:9000 -p 12201:12201 -p 12201/udp -p 514:514 -p 514:514/udp graylog2/allinone

Once that is started, you can now navigate to the instance IP with port 9000 in a browser to get to the Graylog portal. ie. https://1.2.3.4:9000 You can log in with the username and password, which are both “admin”.

Time to add some inputs. Click on “system” up the top right and then “inputs” on the right. If you want a super quick way to add in some fake data, select the input source “Random HTTP message generator” and add it. This will immediately start generating fake HTTP logs to your instance. You can let this run for a while and then pause or remove the source.

Alternatively, you have ports 12201 and 514 passing through data from your machine, so we can use these to set up some data sources. Select the input of GELF HTTP from the dropdown and add the input using port 12201. Then select an input of syslog UDP and put it on port 514.

Some fake HTTP traffic of your own can be added via something like:

for i in {1..10000}; do curl -XPOST http://1.2.3.4:12201/gelf -p0 -d '{"short_message":"Random content '$i'", "host":"testing.com", "facility":"test", "_foo":"bar"}'; done

You can then send some real syslog data via:

On a machine of choice, populate /etc/rsyslog.d/graylog” with the following and restart rsyslog afterwards:

*.* @1.2.3.4:514;RSYSLOG_SyslogProtocol23Format

You should now see data flowing into Graylog with the appropriate searches. You can continue on to create pretty dashboards etc.

Read More

Docker Container Map for Both TCP/UDP

Port mappings from a local machine to a docker container are commonly established as arguments in the ‘run’ command whilst starting a container. By default, mapping port a port like 5000 to 5000 will only pass through TCP traffic “-p 5000:5000″. To make use of UDP, you can use “-p 5000:5000/udp” instead.

To forward both TDP and UDP traffic across that port, you can list both arguments (in more recent versions of Docker). ie:

-p 5000:5000 -p 5000:5000/udp

If you receive an error like the following, ensure you’re doing TCP and/or UDP on a port but not repeating the argument (like assigning TCP twice):

docker failed: port is already allocated

Read More

Workaround for Thunderbird and Icedove Account Setup

For some reason, Thunderbird/Icedove hits a bug when configuring an email account that cannot be auto-configured through the wizard. After attempting to fill in the manual settings, one of the following errors will occur:

Thunderbird failed to find the settings for your email account
Icedove failed to find the settings for your email account

It will then give you the options to “re-test” and the “Done” button will be blacked out. Despite what settings are entered, you won’t be able to progress past this screen.

The workaround is to go back to the first page of the wizard (aka File > Existing mail account). Enter your name and email password and put in “[email protected]” as the email. It will then auto-configure will gmail settings and give you access to Advanced Config. From there, you can either press “Done” and exit the wizard or you can choose to populate the fields with your proposed email server (leave the test gmail address in place).

After that, you’ll then have an account set up and you can proceed to right click on the address, click Settings and replace the gmail address with your proper email and whatever other settings you want.

Read More

Tinder – You’re Out of Likes Workaround

So with the introduction of Tinder Plus, Tinder has now rolled out changes that limit the number of ‘likes’ a regular user can perform in a certain timeframe. Upon hitting this limit, the user is notified that they must wait 12 hours or pay for Plus to “Get Unlimited likes”. However, there is a workaround to swipe without having to wait 12 hours.

Yes, you can just set your phones time forward to trick the app. All the app does is perform a local check on your phone, so setting your phone time to 12 hours ahead (or however many hours it tells you to wait), this will allow you to start swiping again without having to pay for Plus.

Read More

Failure to Pip Install uwsgi

Whilst attempting to install uwsgi 2.x in a virtualenv, the following error is indicative of missing dependency(s):

Command bin/python -c "import setuptools, tokenize;__file__='build/uwsgi/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-uCxZjv-record/install-record.txt --single-version-externally-managed --compile --install-headers include/site/python2.7 failed with error code 1 in build/uwsgi
Storing debug log for failure in /root/.pip/pip.log

Full stack trace:

Exception information:
Traceback (most recent call last):
  File "local/lib/python2.7/site-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "local/lib/python2.7/site-packages/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "local/lib/python2.7/site-packages/pip/req.py", line 1435, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "local/lib/python2.7/site-packages/pip/req.py", line 706, in install
    cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
  File "local/lib/python2.7/site-packages/pip/util.py", line 697, in call_subprocess
    % (command_desc, proc.returncode, cwd))

At least in this case, the following will appease (unless more dependencies are missing):

sudo apt-get install python-dev

Read More

SSH Can’t Connect to Google Cloud Compute Instance

So I go ahead and power up a Google Cloud Compute instance. It doesn’t let me add a key to the VM so I need to wait until the instance is started. Go ahead and add a key. Can’t connect as root, so let’s true their browser ssh. I’m logged in as $myemail@instance. Try to connect as that user? Nope. Using the browser ssh to check out auth.log shows rejection:

Dec  9 17:47:23 instance-1 sshd[3029]: Connection closed by $ip [preauth]

So, I can see that my key has not been copied to ~/.ssh/authorized_keys for $myemail account nor /root/.ssh/authorized_keys . However, a look at /etc/passwd shows that Google has actually created a new unix account, named as the host from the ssh key. Now that the correct user is known, logging in works. Not exactly completely intuitive.

Also, after initially logging into the account and setting up a new “Project”, the following error presented itself:

Error
The project you requested is unavailable.

Then a few hours later the project was able to be manipulated. Also not an insightful message to obfuscate some kind of processing or batched processing occurring.

Read More