IAM-Permissions to allow AWS CloudWatch READ-only (Logs, Alarms, Filter, Insights, etc.)

AWSTemplateFormatVersion: 2010-09-09
Resources:
  User:
    Type: AWS::IAM::User
    Properties:
      UserName: someUser@yourCompany.com
      Policies:
        - PolicyName: allow-cloudwatch-ro
          PolicyDocument: >
            {
              "Version": "2012-10-17",
              "Statement": [
                {
                  "Effect": "Allow",
                  "Action": [
                    "logs:List*",
                    "logs:Describe*",
                    "logs:Get*",
                    "logs:PutQueryDefinition",
                    "logs:Filter*",
                    "cloudwatch:List*",
                    "cloudwatch:Get*",
                    "cloudwatch:Describe*",
                    "sns:Get*",
                    "sns:List*"
                  ],
                  "Resource": [
                    "*"
                  ]
                }
              ]
            }

We use this IAM-Policy to grant CloudWatch-Console access to a given user. The user should be able to browse, filter and use Log Insights on all our logs. We also added logs:PutQueryDefinition permission, thus the user can create Log Insights filter queries himself.

Please note: For this read-only permissions policy, we do not limit to certain resources but take ‚*‘. If you want to restrict further, you should split up the policy actions accordingly and restrict individually.

Update sonar-scanner.properties with version and name from package.json

Lately, I added SonarQube to our build pipeline. SonarQube is a code analysis tool which lets you easily scan your projects for smells and bugs. We mainly use it for Java and NodeJS code. The scanning is straightforward and can, in the simplest case, be started from the developer’s machine by using sonar-scanner. The sonar-scanner checks the current directory for a sonar-scanner.properties-File. In our case, an example properties file looks like this:

# must be unique in a given SonarQube instance
sonar.projectKey=some-unique-project-key
# this is the name and version displayed in the SonarQube UI. Was mandatory prior to SonarQube 6.1.
sonar.projectName=this-will-be-taken-from-package-json-soon
sonar.projectVersion=this-will-be-taken-from-package-json-soon
 
# Path is relative to the sonar-project.properties file. Replace "\" by "/" on Windows.
# This property is optional if sonar.modules is set. 
sonar.sources=.
sonar.tests=tests
sonar.language=js
sonar.exclusions=tests/**, node_modules/**, .*/**, coverage/**

# Encoding of the source code. Default is default system encoding
sonar.sourceEncoding=UTF-8

To simply update the version and package, we start with the script from my other post Keep your swagger.yaml version in sync with package.json and extend it a bit. The result looks like so:

Although this looks a bit overwhelming, the real magic only happens in these few lines.

Put the contents of the gist in a file called updateSonarProps.sh in your project directory, next to the package.json and sonar-scanner.properties and run it via:

sh updateSonarProps.sh

Bonus: to start the scan even easier, I added the following script to the package.json as well:

  "scripts": {
    ...
    "sonar": "sh updateSonarProps.sh && sonar-scanner"
  },

A post on how we integrated that all into our Jenkins will follow soon 🙂

Accessing remote minikube UI via SSH port-forwarding

tl;dr

Get the k8s node’s IP address (the one which runs the dashboard), with:

minikube dashboard --url

Use this IP and do a local port-forwarding, with:

ssh cgiehl@server -L 30000::30000

Today, I was trying out minikube, a simple way to setup a k8s „cluster“ locally and play around with it. Unfortunately, „locally“ didn’t quite work out for me since I was heavily running out of disk space 😀 I’ve heard a lot of ppl and their excitement about running minikube locally, but as soon as you ask them how to quickly run it on a server to mess around, it’s always the same: „On a server you want to setup kubernetes fully!“. Since this was not the answer I was looking for, I took a quick glimpse at my Ubuntu home-server in the corner and gave it a shot!

The installation was super easy and super fast (as expected), however, the moment the official docs stated:

To access the Kubernetes Dashboard, run this command in a shell after starting minikube to get the address:

minikube dashboard

Nothing openend in my SSH login shell 😉 Of course, you can enable X11-forwarding but since the dashboard is accessible via a browser, I wanted to use my local browser anyways. But…

What is the dashboard’s IP?

Now things get interesting and I want to show some ways on how to obtain them. The first thing I tried was

http://192.168.99.100:30000

https://github.com/kubernetes/minikube/blob/09f683bb1d5ea74f649d2829b68227c4ddc8b0eb/cmd/minikube/cmd/dashboard.go#L72-L77

server:~ cgiehl$ kubectl get services --all-namespaces

NAMESPACE     NAME                   TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE


default       kubernetes             ClusterIP   10.0.0.1     <none>        443/TCP         41m


kube-system   kube-dns               ClusterIP   10.0.0.10    <none>        53/UDP,53/TCP   41m


kube-system   kubernetes-dashboard   NodePort    10.0.0.36    <none>        80:30000/TCP    41m

Now we have the exposed port running on 30000. Still we need the IP address!

A search in the repo gave me, that port 30000 is the default port, specified in the service.yaml.

server:~cgiehl$ kubectl describe nodes minikube
[...]
Addresses:
  InternalIP:  192.168.99.100
  Hostname:    minikube
[...]

After figuring that out, I thought this needs to be easier. It turned out, that there even exists a CLI flag for minikube dashboard it (idk since which version).

server:~ cgiehl$ minikube dashboard --url

which directly gives

http://192.168.99.100:30000

Further reading:

.htaccess preventing your webserver to serve sensitive VCS (git, svn, etc.) folders

Just recently I stumbled upon a vulnerability in some websites we are hosting. I don’t usually push these to remote repositories but keep them only in the local git repo to create an additional backup layer in case of breaking updates, etc.

Personally, I like to add an underscore prefix to files which I either don’t want under version control or not being exposed by the http server.

Nice read: https://en.internetwache.org/dont-publicly-expose-git-or-how-we-downloaded-your-websites-sourcecode-an-analysis-of-alexas-1m-28-07-2015/

Keep your swagger.yaml version in sync with package.json

I use Swagger to document my APIs, since I really like the coupling of code & documentation. Also, the NodeJS Swagger-UI, which comes as an express middleware, is really easy to setup and run – bonus: the UI looks very nice!
To render the HTML docs, the Swagger-UI parses the swagger.yaml file you have in your project. However, since also the version is hardcoded in the swagger.yaml it has to be updated manually. How nice would it be if we could use the version specified in our package.json and automatically update the yaml?

Try the script by calling sh updateVersionInSwagger.sh in your project’s root dir and check the swagger.yaml. All correct?

Now, of course you can trigger the script manually, but a more convenient way (which I also use) is to put it in the version script of npm (see docs).
Since we would need the current version number while not yet adding a new git tag, especially the version-hook comes in handy:

Run the version script. These scripts have access to the new version in package.json (so they can incorporate it into file headers in generated files for example). Again, scripts should explicitly add generated files to the commit using git add. (cf. docs)

Finally, we need to add the hook in the package.json as such:

"scripts": {
"preversion": "sh doBranding.sh",
"version": "sh updateVersionInSwagger.sh && git add api/",
"postversion": "git push && git push --tags"
},

Now, you can try it yourself by just typing npm version patch.

Show off your ~/.profiles

I don’t know since when, but I guess starting early on from „How to get access to the university machines (which are of course Linux-only) without looking like a total noob“, I was fascinated about these matrix-like command-line-ninja-people and decided to become on as well (which worked out quite good, I guess!). I can still remember the first lines, the exercise-sheet read:

"ls -la" lists all your files in the current folder
1. Create an alias "ll" for the command "ls -la"
Hint: you can add aliases to your ~/.bashrc [...]

Of course, I thought what the hell is a ~/.bashrc?? Then I found it among the files in my home directory and opened it (second thoughts: how the hell do you edit text files in this linux-bash-thingy? :D).

I finally figured it out, and somewhere down the line you see something like this:

# uncomment for a colored prompt, if the terminal has the capability; turned
# off by default to not distract the user: the focus in a terminal window
# should be on the output of commands, not on the prompt
#force_color_prompt=yes

Yes, I can do that! So you quickly remove the comment hash in front of the force_color_prompt (how quickly depends on your vim-skills of course :)), hit the Esc+:+w+q+!, and restart your bash. Color prompt, success!

Endless possibilities, I wanna be a shell-ninja one day! Of course, back then it ended in a huge amount of non-sense aliases, but some commands still last until today – in my ~/.profiles, on almost every machine I use:

Lastly I have to say, I didn’t stop there…but the tales about my epic ASCII-Art motds is beyond the scope of this post… 😉