≡ Menu

How to strip first few characters of words in Excel ?

At times, you may want to strip first few characters of a word in Excel. For example, consider the following data that shows last names of some random people

Screen Shot 2018-04-23 at 5.33.34 AM

If you just need the first 10 characters of the last names, you can employ the Excel formula LEFT, as shown below.

Screen Shot 2018-04-23 at 5.28.15 AM

The formula is =LEFT(text, number of characters)

After you typed in the formula and pressed Enter, you should see the name HEPPENHEIMER truncated to HEPPENHEIM.

Once your results look good, simply copy the cell (in our case B2) and paste it in all other cells where you need the formula executed. (Excel automatically updates the cell number in the formula as required).

Screen Shot 2018-04-23 at 5.42.13 AM

Beautiful.

Now, you may be wondering if there is a RIGHT command (to strip the characters from right). Well, of course this one. The syntax is the same.

=RIGHT(text, number of characters)

Screen Shot 2018-04-23 at 5.44.39 AM

Nice!!

Notice how space is considered as a character in MAC CIONAODHA.

Tip: If you don’t specify the number of characters, 1 is assumed. (the default value). This means it will strip the first character when using LEFT command and last character when using RIGHT command.

In our example,

For first character: =LEFT(A2)

For last character: =RIGHT(A2)

Screen Shot 2018-04-23 at 5.48.34 AM

There you have it. Let me know if you have any other Excel questions and I’ll try to help you out.

Keep growing.

 

What is Docker? An absolute beginner’s guide

To the uninitiated, we are not talking about Docker clothing company, which makes the popular Men’s Khakis. What we are talking about is the Docker that has changed the way Software Applications are bulit, shipped and run. You have probably heard about Docker and how cool it is, but never really understood fully. And you are itching to finally sit down and read about it. You have come to the right place. In this blogpost, I’m going to demystify Docker for you.  By reading this post fully, you will understand,

  1. What the heck is Docker?
  2. What makes Docker so invaluable and indispensable?
  3. How to install Docker on your PC or MAC?
  4. Most commonly used Docker commands
  5. A full working example

Docker image

Ready? Let’s begin.

What the heck is Docker?

Docker provides a way for Applications to be built and run in a container with all the required software packaged in it. 

But you ask, what in the world is container?

Container is a Docker process that can be run on any Linux or Windows based system. It includes everything it needs to run, including System lirbaries, application code, application dependencies and configuration files. (You throw Docker container in a Car’s bumper and it will still work. Just kiddin.) Containers running on a system share the Operating System Kernel with other processes (including other Docker containers). A Docker container is a mini-machine in its own right.

Tip: You can list all the Docker containers running your system by running the command

docker ps

docker-container

To contrast this with how software applications are traditionally run, look at the image below.

docker-container-1

At this point, you may be wondering, ‘Wait a minute. I’ve seen this before. Are you not talking about virutalization? Vmware and stuff?

Not really. Simply put, vmware virutalizes the underlying hardware resources. But Docker virtualizes the Operating System that runs on top of Vmware virutal machine.

docker-container-vm.png

At this point, if you have been doing Application Support for a while, one stricking advantage should be obvious to you: Consistancy of environments. Think about how many times you have been told by the development team: ‘Oh, but it works in my local Dev enviornment. Something must be wrong in production servers. May be a jar file is missing in the classpath in Prod?’ Painful. Docker puts an end to all this environment specific mysteries.

So, to summarize: Docker is a container solution that enables buliding, shipping and running applications with all the required software in a single unit. The benefits include consistency across deployments, fast startup, flexible and developer-friendly build process.

Enough fluff. Let’s get our hands moving by actually running a Docker container.

Installing Docker

Before you can actually run your first Docker Conainter, you have to install Docker software.

What do you need to begin?

A Linux or Windows or MAC system (can be a desktop or a server).

There are two flavors of Docker. Community Edition and Enterprise Edition. Community Edition is free and mainly greared towards learning and testing purposes. Enterprise Edition comes with few bells and whistles and also comes with an invoice in the mail. We will be using Community edition to learn Docker. (Note: You cannot install Docker Enterprise Edition in Desktops). For full compatibility matrix, check this link.

Note: In older MAC and Windows systems, Docker was installed using Docker Toolbox, a legacy desktop solution that inlucded docker-machine, docker and docker-compose commands (more on these commands later). If you have newer MAC ( OS X El Capitan10.11 or later) or Winodws (Winodws 10 or later) It is recommeded to install Docker for MAC or Docker for Windows instead of Docker Toolbox.

MAC

To install Docker for MAC, download the dmg from Get Docker for Mac (Stable).

Double click Docker.dmg. Drag and drop to the Applications folder as shown below.

docker-app-drag

Note: You must have Admin privilege to perform the install, as shown by the pop up below that appears when you run the install

Screen Shot 2018-02-17 at 10.09.09 PM

Once installed, the docker icon (Whale) should appear on your top status bar.

Windows

Download the installer from Get Docker for Windows (Stable).

Simply launch the exe file and follow prompts. It should be a super straight forward install.

Ubuntu

You can use apt-get to install from the Docker repository or install a deb package using dpkg.

Follow the instructions here

CentOS

You can use yum to install from the Docker repository or download rpm from here

For detailed instructions, go here.

Great, now that you’ve installed Docker, let’s start playing with it with some basic commands.

Using Docker for the first time

We will be using Docker for Mac to illustrate. The commands are exactly the same in other flavors.

First thing you want to do is to make sure you have the latest version of Docker (at least not some ancient version). As of this writing, Docker CE 17.12.0 is the stable version. Click on the Docker icon (Whale) at the top status bar and click on About Docker

Screen Shot 2018-02-17 at 10.34.39 PM

Open a terminal and type the following command

~$docker version
Client:
 Version: 17.12.0-ce
 API version: 1.35
 Go version: go1.9.2
 Git commit: c97c6d6
 Built: Wed Dec 27 20:03:51 2017
 OS/Arch: darwin/amd64

Server:
 Engine:
 Version: 17.12.0-ce
 API version: 1.35 (minimum version 1.12)
 Go version: go1.9.2
 Git commit: c97c6d6
 Built: Wed Dec 27 20:12:29 2017
 OS/Arch: linux/amd64
 Experimental: false

Notice the GO version. Docker is built using the Go programming language.

If you simply type the command docker and press Enter, you will see nice concise command list.

Screen Shot 2018-03-12 at 8.01.09 PM

If you need help with any of the commands, simply type the following command:

docker <command name> –help

~$docker ps --help

Usage: docker ps [OPTIONS]

List containers

Options:
 -a, --all Show all containers (default shows just running)
 -f, --filter filter Filter output based on conditions provided
 --format string Pretty-print containers using a Go template
 -n, --last int Show n last created containers (includes all states) (default -1)
 -l, --latest Show the latest created container (includes all states)
 --no-trunc Don't truncate output
 -q, --quiet Only display numeric IDs
 -s, --size Display total file sizes

Okay, let’s get down to business. Let’s actually run a docker container.

With docker, there is this concept called images. First, you create an image and then you run the image using a container.

Image ——–> Run ——–> Container.

Running your first container

From the shell prompt, execute the following command.

~$docker run -i -t ubuntu /bin/bash
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
d3938036b19c: Pull complete
a9b30c108bda: Pull complete
67de21feec18: Pull complete
817da545be2b: Pull complete
d967c497ce23: Pull complete
Digest: sha256:9ee3b83bcaa383e5e3b657f042f4034c92cdd50c03f73166c145c9ceaea9ba7c
Status: Downloaded newer image for ubuntu:latest
root@a5dfc81d2340:/#

Let’s disect the above command.

docker: This is the docker executable

run: This tells docker to run a container

-i: This tells docker to keep STDIN open (i.e to be interactive)

-t: This tells docker to allocate a tty ( so that you get a nice terminal to type commands in)

ubuntu: This is the all important parameter – this tells docker which image to run. If the image is not available locally, docker will try to obtain the image from docker repository (hub.docker.com)

/bin/bash: This is the command to be run when the container begins running.

Open another terminal window and type the following command:

~$docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a5dfc81d2340 ubuntu "/bin/bash" 3 seconds ago Up 5 seconds practical_kepler

docker ps shows the containers that are currently running. Notice the container id. Also notice how the docker command borrows ps which is unix command to show the runnign processes.

Note: you can use the command docker ps -a show the containers that had run in the past but no longer running.

~$docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ubuntu latest c9d990395902 3 days ago 113MB

docker images shows the avaialble images. When I ran the docker run command, it pulled the ubuntu image from hub.docker.com. Note the tag latest. Tags are how you keep various versions of the image. If no tag is specified, the defaut tag would be latest.

Digging deep

So, what is the nature of the container I just started ? Let’s run few commands in the container’s bash shell that was started earlier.

root@a5dfc81d2340:/# cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.4 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.4 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

So, what you got was Ubuntu 16.04.4 (from the version field).

Note: What if you want a sepcific version of ubuntu ? You need to go to hub.docker.com and check the available ubuntu versions.

Screen Shot 2018-04-16 at 7.37.37 AM

When you search for ‘ubuntu’, you get the following images:

Screen Shot 2018-04-16 at 7.38.23 AM.png

Notice the ‘Supported tags and respective Dockerfile links’. Note how 16.04 is tagged as latest. If you want ubuntu 14.04 for example, your docker run command sholud look like the following:

~$docker run -i -t ubuntu:14.04 /bin/bash
Unable to find image 'ubuntu:14.04' locally
14.04: Pulling from library/ubuntu
c2c80a08aa8c: Pull complete
6ace04d7a4a2: Pull complete
f03114bcfb25: Pull complete
99df43987812: Pull complete
9c646cd4d155: Pull complete
Digest: sha256:b92dc7814b2656da61a52a50020443223445fdc2caf1ea0c51fa38381d5608ad
Status: Downloaded newer image for ubuntu:14.04
root@2cbd08aaa731:/#
root@2cbd08aaa731:/# cat /etc/os-release
NAME="Ubuntu"
VERSION="14.04.5 LTS, Trusty Tahr"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 14.04.5 LTS"
VERSION_ID="14.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"

docker ps will show you that you are running 14.04

~$docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2cbd08aaa731 ubuntu:14.04 "/bin/bash" 4 minutes ago Up 4 minutes fervent_kepler

Yes, it’s that simple.

Coming back to the ubuntu 16.04 container we started earlier, let’s run couple of commands in the container.

root@ff8b7428adf6:/# df -m
Filesystem 1M-blocks Used Available Use% Mounted on
overlay 64251 1447 59512 3% /
tmpfs 64 0 64 0% /dev
tmpfs 1000 0 1000 0% /sys/fs/cgroup
/dev/sda1 64251 1447 59512 3% /etc/hosts
shm 64 0 64 0% /dev/shm
tmpfs 1000 0 1000 0% /proc/scsi
tmpfs 1000 0 1000 0% /sys/firmware

df -m shows the file systems that have been mounted with their sizes in megabytes.

The beauty of docker is, all these file systems are completely isolated from the host operating system. It’s as if you are sitting in a OS inside an OS. Whatever you do to this container

Splunk vs ELK

If you are in IT Operations in any role, you have probably come across either Splunk or ELK, or both. These are two heavyweights in the field of Operational Data Analytics. In this blog post, I’m going to share with you what I feel about these two excellent products based on my years of experience with them.

The problem Splunk and ELK are trying to solve: Log Management

While there are fancier terms such as Operational Data Intelligence, Operational Big Data Analytics and Log data analytics platform, the problem both Splunk and ELK are trying to solve is Log Management. So, what’s the challenge with Log management?

Logs, logs, logs and more logs

 

The single most important piece of troubleshooting data in any software program is the log generated by the program. If you have ever worked with vendor support for any software product, you have been inevitably asked to provide – you guessed it, Log files. Without the log files, they really can’t see what’s going on.

Logs not only contains information about how the software program runs, they may contain data that are valuable to business as well. Yeap, that’s right. For instance, you can retrieve wealth of data from your Web Server access logs to find out things like geographical dispersion of your customer base, most visited page in your website, etc.

If you are running only a couple of servers with few applications running on them, accessing and managing your logs are not a problem. But in an enterprise with hundreds and even thousands of servers and applications, this becomes an issue. Specifically,

  1. There are thousands of log files.
  2. The size of these log files run in Giga or even Terra bytes.
  3. The data in these log files may not be readily readable or searchable (unstructured data)

Sources_of_logfiles (4)

Both Splunk and ELK attempt to solve the problem of managing ever growing Log data. In essence, they supply a scalable way to collect and index log files and provide a search interface to interact with the data. In addition, they provide a way to secure the data being collected and enable users to create visualizations such as reports, dashboards and even Alerts.

Now that you know the problem Splunk and ELK are attempting to solve, let’s compare them and find how they are achieving this. I’m going to compare them in 4 areas as follows:

Technology

Cost

Features

Learning Curve for the operations team

Got it ? I can’t wait to share. Let’s dive in.

Splunk_image

VS

ElasticSearch Logo

Read More

8 useful shortcuts for lightning fast web browsing

Let me share with you 8 super easy shortcuts to speed up your web browsing. As with any software tools, learning keyboard shortcuts can save lots of time and enables you to look like a pro in front of others 🙂

1.Bookmark a page with Ctrl D

While on the page you would like to bookmar, simply press Ctrl D to bookmark the page. You can choose a folder to save the bookmark. I like to organize my bookmarks by creating various folders.

Screen Shot 2017-11-11 at 9.36.40 PM

2.Open a new Tab using Ctrl T, close using Ctrl W and switch using Ctrl Tab

While on a page, simply press Ctrl T to open a new tab. Note that you can use Ctrl N to open a new browser window. I personally like to open tabs instead of new windows.

Screen Shot 2017-11-11 at 9.41.13 PM

To close the current tab, simply press Ctrl W.

To switch between tabs, press Ctrl Tab to jump to the tab on the right. If you need to switch to the tab on the left, simply press Ctrl Shift Tab

Note for Mac users: You have to use Ctrl key on your MAC for this shortcut (Not the Command key). On all other shortcuts, use Command key instead of Ctrl key.

3.Jump to the URL field using Ctrl L

Wherever you are on page, clicking Ctrl L takes you the URL field. You can immediately start typing the url you wnat to visit.

4.Scroll down a page using Space bar

This can be very handy if you hate scrolling down using mouse (either using a scroll wheel or using the side bar). Simply press space bar to scroll down one page. As you may have guessed, simply press Shift Spacebar to move up one page.

Read More

So we all know that Java Heap is a crucial resource, lack of which will kill your application. Naturally you will want to monitor the heap usage. Bur surprisingly it is not very straight forward to measure the heap usage of your JVM unless you have a modern APM (Application Performance Management) tool implemented. To make things worse, in Windows world, the memory you see in Windows Task Manager (a solid tool by btw) is NOT the same as the JVM heap size.

For example, let’s say you have set maximum java heap at 3GB. But it is quite possible that the memory shown by Task Manager be higher (much higher at times), say 3.5 gb or 4 gb. You can pull you hair trying to figure out where that extra memory utilization came from, or you can read rest of this article and put an end to the mystery.

Generally speaking, here is the reason memory shown by the Task Manager is more than the Heap: The memory shown in Task manager is the entire Memory footprint of the JVM and NOT just the Java Heap of the JVM. Note that the JVM is just an another process as far as Windows is concerned.

Read More

What you didn’t know about java.lang.OutOfMemory Error!

Few errors are deadlier than java.lang.OutOfMemory errors. When this happens, you application will run into a very unpredictable state, often hanging up and not processing new requests, coughing up ugly stack traces into users browsers etc. The most popular (and effective in the short run) fix for OutOfMemory error is simply restarting the Application or Application Server.

While running out of java Heap is the most common OutOfMemory error, there are indeed several types of OutOfMemory that can occur. In this post, I will show you these various types of OutOfMemory errors and what they mean.

java.lang.OutOfMemoryError: Java heap space

The most popular one. This is when JVM goes belly up and unable to allocate memory in Heap . Your application might appear hung up and extremely slow to respond to user requests.

For example, let’s say your Java application has 2GB of memory maximum (through -Xmx flag). When the entire 2GB is used up and GC is unable to reclaim any memory, the next memory allocation request for an object will fail with java.lang.OutOfMemory Error.

What causes java.lang.OutOfMemoryError: Java heap space ?

Read More

One important change in Memory Management in Java 9

Java 9 is supposed to be released in summer 2017. The biggest changes appear to be around modularization which enables  you to create, build, deploy and reuse modules of code. There is also the new jshell command line tool that lets you test snippets of code quickly. From Application support standpoint, I see one major change in memory management and that’s what I want to discuss today.

GC (Garbage Collection) is the bread and butter of Memory Management in java. GC  is responsible for cleaning up dead objects from memory and reclaiming that space. GC executes its cleanup using predefined Collectors that use certain algorithm(s). As I mentioned in my previous post, there are 4 Collectors that can be used.

  • Serial Collector
  • Parallel Collector
  • CMS (concurrent Mark and Sweep) Collector
  • G1 Collector

Until Java 1.8, on Server class machines (2 CPU and 2GB RAM minimum. Humor me if your server is leaner than this size), the default collector is Parallel collector.

Parallel collector is all about throughput. It does not care about having longer GC pause times (as it will stop the application threads before performing a GC). If you cannot tolerate long GC pause times (upwards of 1 second), you could choose CMS collector.

G1GC was introduced in Java 1.7 to address both increased throughput and decreased GC pause times overall.

Here is the major change with this edition of Java. G1GC is the default collector in Java 9.

Read More

In this post, I’m going to explain how Heap is organized using generations and how Garbage Collection works behind the scene to free up memory. Java memory management has evolved a lot over the past few java releases.  Understanding the mechanics underneath will help you better tune it (if required) to suit your needs.

When your java application runs, it creates objects which takes up memory space. As long the object is being used (i.e referred by the application somewhere), it is going to occupy the memory. When the object is no longer used (for example, when you cleanly close a DB connection), the space occupied by the object can be reclaimed by Garbage collection.

What are generations ?

Java heap is typically divided into two major pools. An area where short-lived objects live. And an area where long-lived objects live. Young generation (aka Nursery aka Eden) is the place for short-lived objects and Tenured generation (aka Old) is the place for long-lived objects.

Screen Shot 2017-05-23 at 7.20.32 PM

Why Generations ?

Read More

How to monior Heap usage of a Java Application ?

Heap is the memory space where java application objects are stored. Unfortunately the size of Heap is limited (by the -Xmx java command line option). When all of heap is used up, you are in deep trouble. So, it helps to understand how to measure Java heap usage, and that’s exactly what I’m going to be discussing with you in this post.

Very surprisingly, measuring heap is not that straight forward in java world. I believe it is due to the fact there are numerous ways to measure java heap. Oracle seems to introduce new tools with every release (and retire the old ones). For this reason, I recommend everyone to invest in a commercial grade APM (Application Performance Management) tool. With APM tool, you will simply login to the APM interface (typically a web application) to view the JVM heap usage (along with tons of other useful metrics)

Using the command jcmd to measure heap usage

Java comes with a command line tool named jcmd. It should be available in most flavors of Java (Oracle, IBM etc). I’m going to recommend jcmd to measure your heap usage. One of the advantages of using jcmd is the class histogram which not only shows the heap usage but it breaks down the usage by class. This gives you an instant view of what is causing heap to fill up.

Important Note: jcmd does have some impact on the jvm. But I assure it is worth the price, as the details shown by jcmd are very valuable.

Important Note: the command jcmd must be run as the same user (or effective user) as the user running the java application

Let us see jcmd in action.

Read More

As an Application Support Engineer or a Lead Developer,  you are supposed to save the day when things hit the fan. But this is easier said than done.  I’ve dealt with these issues on a daily basis for couple of decades. In any enterprise application, there are 4 major subsystems that cause performance degradation or outage. Even though the Application platforms have drastically changed in the past few years, these 4 subsystems still make up majority (if not all) of the issues. Understanding this obvious, yet often overlooked fact, you will be able to ask intelligent questions during troubleshooting.

Without further due, here are those:

  1. Application Serving Platform (typically an Application Server or PAAS)
  2. Backend Database or other Remote service(s) that your application depends on
  3. The Operating System that hosts the Application
  4. Application Code

And no, Network is NOT one of the major subsystems to point finger at. That is not to say that Network issues do not occur at all. But in practice, I would look at Network when all other suspects have been eliminated.

Also note that the 4 subsystems listed are not in any order. That is, they all have equal chances of being the culprit.

Image: 4 Subsystems responsible for Application stability

4 Subsystems

(C) Karun Subramanian

Read More