Saturday, September 26, 2015

Getting Older or Getting Better?

I've crossed 40 lately. In hi-tech I am officially old. Not really old yet, this is reserved for 45-55 guys. Nobody talks about *those* guys. They can really see the retirement around the corner.

As I am getting older, there are more and more conversations with my colleagues about staying relevant and how hard it is to find a next gig. Apparently, they are getting older as well and worried about the age. 
I am not worried about find a job. I have a secret that keeps me in a loop. 
I am a batman. Wait, this is my other secret...
My secret is that I practice. 

Why people are afraid that it will be hard to find a job at 40-50+? 
Because they are afraid that 20-30+ folks can be hired instead of them at lower price to do the same job. They are right. 
Why should anyone hire super senior developer, when he can get the same level developer paying half as much? 
Job market behaves like any other market, i.e. it is driven by supply and demand. If demand it higher than supply then the job is in ``shortage''. If demand is lower than supply then the job is in ``surplus''. Clearly, it is easier to find a job if your skills are ``shortage'' skills, i.e. you can fit ``shortage'' jobs. Interestingly enough, this is correct for any level in a given skill.
It is tempting to say that good kernel hacker will find a job much faster than the average kernel hacker. This is not really a case. Good kernel hacker will not turn to jobs that average hacker will be happy with. This is true both in the quality of tasks she will get and in money she will earn. Those hackers basically do not compete for the same jobs and thus, good kernel hacker might be in surplus market while average might be in shortage market.

If you are 40+ old, the last thing you want to do is to compete with all the young folks for the same job. Maybe not the last, the last thing will be to wrestle with a bear, but second last. This means no PHP or Angular for you. Unless, you are writing those things. You cannot just be an Angular programmer, you have to be Angular expert. The one the young developers are coming to ask really hard questions and who's presentations they are watching to learn Angular. You should either be the first one running with the thing, this means investing your time into getting better and also putting some of your stuff out there, or you should go deep. Really deep. This again, requires investing your time into getting intimately familiar with Angular (or anything else you pick). 

Another possible way is to pick some niche or a field and invest a lot of time in it. I was picked by Storage somewhere around 2000. Since then I've worked in a few companies on a different things, I have read storage news, listened to storage startup speeches, watched videos, read books and blogs. Only about 1% of all this information stays in my brain, but over 15 years this is enough. In storage, I have a handicap over younger folks, which is a net gain for storage companies and for me. Teenagers are still way cooler and looking better, so I will not get all jobs. 

There are plenty of niches out there. For example, I have interviewed a lot of a young developers and most of them do not know C/C++. Come to think of it, almost nobody knows C++. Almost no young developer understands kernel development. Especially Windows kernel development. Interestingly enough, most of the kernel "experts" do not understand it either. Build and deployment areas are currently undergoing a major revolution with DevOps and automatic tools. Most of the developers never heard about it. Distributed systems are still hard for humans, even with the abundance of messaging queues and service buses. Clustering is a high-promise concept for 20 years and it is still hard. Any software that is close to HW is good. Anything that requires holistic view is great. There are plenty of others, just choose.

So, pick something and get better in it. Don't just "work with it", but get better. Read, learn, improve and repeat. Do not experience the same year over and over again. If you actively improve yourself, the equation flips over. Now the time is not your enemy, it is your friend. Well, not friend as "lets have a beer together after work", but the time is working for you. All of the sudden, you are not just old, you are experienced. Moreover, the older you are, the more experienced you become.

Friday, September 11, 2015

Making graphs for your scientific paper

I am writing a paper that compares a couple of algorithms. My prof told me to write a simulation and to make some graphs. As this is not the first time I am doing that, I wanted to describe the process. The first was painful. This time I almost enjoyed it.

The first time went like this. 
  1. Implement a simulation. There is a main class that drives the simulation and then there are specific algorithms. So, the main class calls each algorithm and the algorithm prints out the results.
  2. Redirect the results into a file. 
  3. Open the file in your editor of choice, which is clearly VIM. Manually combine columns into a single file that gnuplot can understand.
  4. Open gnuplot and draw a graph. Save the graph in PNG and embed in a paper.
Now, I send the paper to my supervisor and gets a lot of rejects. Dataset should be larger, samples should be smaller, parameters should be different, etc. So, it is back to the same 4 easy steps.Rinse and repeat. And repeat, and repeat...

The process should be automatic. This is how I did it this time.

Gnuplot should get data files that have columns. 


To eliminate step #3, this file should be output by simulation directly. The problem is that each column is generated by a different algorithm, which are called sequentially by the code. The way to do that is counter intuitive in a way, the results have to be gathered by the main class, which will then output them all together. 

 This is a code snippet. Gathering of results are highlighted in yellow. Notice that not only results are gathered but also the titles of the columns are fed from the specific algorithms to the main. This is absolutely required, as each line in gnuplot should be labeled and I don't want to add those labels afterwards.

The next step is to automate graph generation. This is done by outputting a gnuplot script (outlined in blue). So, now simulation runs, outputs data into file of all algorithms, outputs gnuplot script. Gnuplot then generates graph automatically.

The best part is that the code generates two graphs in a single run (two calls to "OutputGnuplot"). I can easily generate another one, as all the data is in one place.

As I said. Almost enjoyed it.

Sunday, March 01, 2015

"Agile is fun, waterfall is not fun" ? Meh...

I've got this comment for one of my posts: "Agile is fun, Waterfall is not fun". This was such as terrific comment that I just had to respond with a post! It is terrific because: it is so wrong and so common at the same time!! It is marvelous!!!

Many people think that agile is fun, while waterfall has a reputation of previous century technology. All the cool kids are doing agile. Old timers are doing waterfall. Boooooring. 

What do you want to be? Don't answer, this is not a real question. The real question though: Is Agile really more fun than Waterfall? 
All things can be proved with fabricated example, so bare with me.  

You are looking for a job and have two options. "CoolGuys" company is developing messenger application for mobile. They have no customers, the idea is rather stupid in 2014, but they are working in Scrum. They have a certified Scrum master, stand-up daily meetings, estimation poker and all the Scrum stuff. Your second option is with "JustHereToWork" company. Most of the development team is over 35 (read old timers). Most of them are smart, apart from a few guys who are extremely smart. I mean Turing award or world-wide name smart. They have a great idea for a product, that is already becoming a major hit and they have the ability to make it happen. They also work in waterfall. Which company is more fun to work at? I would say the second. 

Let's say that the product is important and is a large part of a "fun" at work. What else?

Continuing with the example, you find a couple of friends that have worked at each of those companies and call them. Bill, the guy that worked at "CoolGuys", says that behind the "we are cool guys having fun" facade, the interpersonal relationships are very bad there. People don't help one to another and most of them don't really like one another. Even though the company organizes pizza evenings and game nights, the guys still hate one another guts and fighting over everything. Then you call the second friend, Joe, that worked in "JustHereToWork" company. The guy moved to another city and thus, had to leave the work place. For more than an hour he talked about the great guys at his previous company and how they have met at bar or on a weekend. They are still friends and even some folks has come to visit him in his new home. Hmmmmm. 

So, "fun" is more related to people than to development method. What else?

You call Bill once more, as you think that maybe the interpersonal problems were caused by Bill's personality. After all, you are such a great girl/guy, for sure you will get along with all the folks there. Bill explains to you that even though he had 10 years of experience working at ICQ/MSN messenger/Yahoo messenger and bunch of other programs, still his responsibility at the company was very limited. This is not only him, as all other guys had to go through loops to get any code into a product. This is even before the company have customers. Now, this is a bummer, as you like having a lot of stuff to work on. You call Joe to check the situation at "JustHereToWork" company. For sure, with all the smart guys there Joe was in charge of a single class method. Surprisingly, Joe tells you that he was a master of his domain. He has a lot of responsibilities there and was the final authority in a number of things. 

OK. Responsibility is another "fun" thing at work. Is there anything else?

When you go to sleep, you suddenly remember how Bill complained about being stuck at the same place and nowhere to advance. Apparently his boss didn't trust anyone and the company executives didn't think that that employees development is a good thing. Actually, they didn't think about it at all. Next day you call Joe and ask him about development path in his previous company. What a mistake that was... Joe talked for half an hour without stopping about different options, mentioned discussions with his manager and company policy. His final words were: "you only have to choose what you want to do!" 

This is not a fun thing to be stuck at one place, so lets add development plan to something that is fun at work. Anything else?

Is it fun to work on "yet another messenger"? Is it fun to work on a product that will be forgotten sooner than the company ships third version? Isn't it more fun to work on something that changes the world? If it changes the world for a better, this is even more fun! 

Impact is (yet) another item on my list of fun things. I don't know about you, but I want to change the world with my work. This is what I call fun. 

Is there anything else? Yep. Is development process part of the list? Nope.

Saturday, February 14, 2015

The race is over! The fastest development process is ... not Agile!

If I had a dollar for every time somebody says: "We have to deliver fast, lets go Agile", I would have about 20$ already. This does not sound like much, but this is the REAL number of times I have heard it. Moreover, nobody has ever said: "We have to deliver fast! Lets do good old waterfall !!!"

Well, are all those people wrong? YES! 
The results just came in. Agile process is not the fastest development process. Who come in first you ask? Waterfall.

How come? It is actually simple. Lets say that you have 100 boxes that you have to move to another place. You have a team of 10 completely identical people at your disposal. They are identical just to make the calculations easier, not because they are cloned or something like that. Every person can move one box in one minute.

How agile development, sorry, box moving looks like. 
  • Short sync and planning (1 minute): you take this box, you take this box and etc. 
  • Move the box - 1 minute.
  • Retrospect (1 minute): how was your moving? Any insights?
  • Repeat until all boxes moved. 
There will be 10 such iterations, 3 minutes each. In total this will make 30 minutes. 

Now, lets do waterfall. 
  • Long planning (10 minutes): you take those 10 boxes, you take those 10 boxes and etc. 
  • Move the boxes - 10 minutes.
  • Retrospect - 1 minute.
Total time: 21 minutes. 

Now, if you think that this example is not fair, then you are right. 
Usually, agile will spend much more time shuffling already moved boxes as they are not in the right place. Yep, agile processes excel at re-work. Since the goal is to make the minimal working version at each iteration, many times it is not the optimal one. Then it has to be re-structured and re-done. This will add more time, but the process is enjoyable and team is having a great time at all those stand-up meetings. 

Why do agile then? 
The answer is simple: because you want to be agile (this is me trying to be funny). Notice: not fast, agile. Lets say that 12 minutes after the start product manager decided that boxes have to be moved to a different location. 
  • Agile process can adjust on the very next iteration, i.e. every 3 minutes. Total time will be: 12 minutes (before the change) + 30 minutes (to move the remaining boxes and those that were already moved) = 42 minutes
  • With waterfall you will have to re-start the entire process when it ends. This is basically because there is no process to stop the waterfall. Total time will be: 21 minutes (to complete the first iteration) + 20 minutes (retrospect and blaming one another) + 21 minutes (move the boxes to a new location) = 62 minutes.

Sometimes, being agile actually gets you faster to the ending point. This happens when requirements keep on changing or when you are running in zig-zags under fire. 

Saturday, January 24, 2015

Developers Guide: Why do we need Product Manager?

If you are a developer, I know what you are thinking: "We don't need them! They only make noise, create meetings and do no work!"
Well, you are wrong. They do make a lot of noise, create meetings and do not code. BUT, we do need them. 

First some (mucho little) background knowledge about Product Manager job, whom I will call PM in this piece. There are two kinds and two types. 
type \ kindGood Bad


While everybody can tell Good from Bad, Inbound and Outbound require explanation. 

Inbound PM usually comes from development. He is a technical guy and feels at home with the architecture. Thus, the prefers to define to engineering how the product looks, behaves, what it does, how the features work and how many concurrent users it should support. Such PM works closely with the engineering and writes detailed specs from which architecture and test strategy/plans are derived. Writing those specs is a LOT of work! Microsoft has come with some magic number of 1 PM per 3 Developers. However, their PM's are also program managers, so you could get by with 1 PM per 5-6 developers. 

Outbound PM can be recognized by the spec he provides. If the spec is an Excel file with one-liner per feature, then this is an outbound PM. Such PMs work with marketing, sales, professional services and customers. They concentrate all the input coming from the outside, analyze it and digest in PRD. Clearly one person writing such specs can provide a work for gazillion of engineers. Just write: time machine and teleportation. Done. BUT, gathering all this input, analyzing and deciding what is important and what is just a junk also a LOT of work. Still there is no need for 1:5 ratio, it easily can be 1:20 (depending on the project). 

Clearly, this is a simplification, as just like people are not pure optimists and pessimists, PMs cannot be clearly cut into inbound vs. outbound categories. Most of the PMs are somewhere in between with some strong tendency towards one of the categories. This tendency defines the focus, effort, amount of invested time and consequently, deliverable content. In that order. Here actually lies a problem with PMs and this is the main distinction from optimist vs. pessimist separation. Wake up, this is the important part: 

I (just like 99.999% of the people) don't need to decide whether I am an optimist of a pessimist. It does not matter to me and I don't care. PM, on the other hand, has to decide what type of PM he/she wants to be: inbound or outbound. He cannot be both and he cannot move from one side to another. Just like I cannot be both optimist and pessimist. 

The problem is that some PMs do not decide, so they belong to the new category: "Not decided yet". 
This leads to the following table and I will fill it in as well. There are no "pros" in "Bad" column for obvious reasons. 

type \ kindGoodBad
InboundPro: the product advances really good and fast, and the product is very consistent to the users.
Cons: the users might not buy the product as it does not solve their needs
 Developers ignore those PMs, but they keep on trying to define low-level things, there are constant clashes between PM and Dev.
Not decided yetPM is able to pull a little bit of everything. If Dev Manager is good and can do Inbound work (or at least part of it), then things will work out. If Dev Manager is not good, then PM will quit and the fate of the project will depend on the next PM.There is no direction and no customers. PM and developers  both clash one with another on daily basis (when PM decides to be inbound) and blame one another. 
OutboundPro: the product does what customers want and solves real problems.
Cons: The product looks like a patch-work. There are no conventions in UI, names or behavior. Each developer decides on something else and it shows. 
Nobody wants the product and it also does not look good. PM and developers blame one another. 

To summarize this already long post: you have to decide whether to go with an affair or a spouse, and then do your best there. The indecision period is very risky.  

Friday, January 23, 2015

Career Advice: Stop Whining and Do Your Drill

"I feel stuck man. Not sure what potential I have at my current job. I just not happy here. People are great, but something is missing...

This is not the first I hear that, so I know exactly what to do. Usually I slap the guy, but this one looks like he could slap be back, so I smoothly move to the verbal part:

"Stop Whining and do your drill"

We, read as "software engineers", were trained to build products. Why your career is different? Do you know how to build a product? If yes, then you know how to build a career. Before I give you a recipe there is one catch. 

Career is not only climbing a corporate ladder. I have decided a long time ago that I want to work only on interesting stuff and made it a goal. Lately, I have noticed that I am good at building a new stuff, so I want to concentrate on that. One of my ex-co-workers decided that he wants to be happy and be home at 17:00. It can be anything really, not only "become a VP".

Now the plan:
  1. Define where you want to be in 15-20 years. 
  2. Divide it into 3 year chunks. 
  3. Think how you get to the first milestone.
  4. Get to work.
Oh, you don't know where you want to be in 15-20 years? Well, there is a different method then. Designed especially for guys that don't know what they want in a long-run: Agile Development. How does it look in Agile? Simple.
  1. Define your goals for the coming 3 months. This should be easy as the goals are simple. They don't even have to be career/development goals. You can say: "I want to learn something new", "I want to feel that I've done something" or "I want to start some new project". It can be anything that you can somehow know whether it happened or not. I suggest to record your current state when you do that for comparison. You should also think about the goals and how to achieve them.
  2. After 3 months check where you are. If you have achieved your goals, then return to step #1.
  3. If after 6-9 months you are still unhappy, then you have to seriously re-evaluate your goals. There is something wrong with them or with how you achieve them. 
  4. If after 9-12 months you are still unhappy despite all the above. Start looking for new options. It is not worth it. 
  5. If after 2 years you are still unhappy and at the same place, come to me and I will slap you. 
Good Luck!

As a side note, the company I work for (EMC) has a quarterly goals system. It is mandatory, like in many other places. This system actually can be used for the agile method. It also will give you a feeling of purpose doing it.