The art of mastering

Everyone wants experts, no one like mediocre. So what makes one an expert? Simple – deliberate practice. Malcolm gladwell, in ‘Outliers’ says that it takes 10000 hours to master any subject. How about programming? A person doing a 40hr week does around 2000 hours of programming, so that is 5 years. Hence the reason, most job positions ask for 5-9 yrs of experience. As simple as it sounds, it is disheartening to know that it is not true. I have seen programmers with 2-3yrs experience out perform their seniors. How is it possible then?

The answer lies in the fact that programming is a thought process; it is not a tangible entity. Once you know and believe that you can program, you need to apply it. The application is the process of mastery. Good programmers are good because, they spent time solving problems by pondering. The process of programming is not a straight forward verb. It is thought process that involves coming up with solutions or looking out for alternatives; and understanding them so that one can apply them in their truest form (not by pasting the code found elsewhere).

Good programmers try to minimize their technical debt (described by martin fowler). They try to be hands on. Most good programmers have the following traits.
1. Understand problems, and categorize them into known and unknown (or solvable on their own or need help)
2. Try to devise more than one solution, how so ever bad the stupid solution be
3. Discuss the hard case solutions with others to seek more inputs. (like a riddle/puzzle)
4. Read books (and gain insights from others who had been through the same) a smart cut approach.
5. Learn different programming paradigms (not languages aka syntax)

Ever wondered what is the difference between a programmer and a designer/architect? A good programmer is like a artist. Given a tough job, the goal of a good programmer is to realize it in its spirit. Good programmers circumvent problems and take solutions that burden them the least. A designer is a mature programmer, One can call oneself a designer when one devises a goal and achieves it.

Designers’ genuinely discover themselves to be so. They set forth goals and achieve them in a way that acts as an example to others. One can never be a good designer by virtue of a title or experience. Design is a discovery process, unlike programming. The true benchmark in my opinion is when a designer devises a solution all by himself and finds out that it is in fact a standard/recommended practice. Designers are above the implementation details and should realize that by achieving things the way they want , and not by the way the obvious solutions allow.
If a designer sets goals and cannot achieve them, then it is inadequate. Good designers preach and practice, not to substantiate their gospel but to discover new means.

Behind the interview door

Getting a job is a momentous occasion. No matter, how much you yearn - you always go through the dreaded interview phase. Now don’t get me wrong. I am not saying that interviews have to be miserable. I am used to sitting on either side of the desk. While it is a joy to interview candidates who fit the bill; It rarely happens. No amount of preparation can guarantee a success. Interviewees aspire to go by the ‘book’ (answering everything under the sun). However good interviewers don’t go by the book, they are great listeners. I had many a great opportunity to share the hall with some of them and I found the lessons invaluable. I am writing this for the benefit of many, with whom I shared and will share the experience.
While CV may say many things, there are certain things which become obvious only during the human interaction. The most prominent ones are below (in no particular order). The traits that make an interviewee most endearing are the ones I listed below.

Confidence/ Humility:

Do this by the correct measure. Don’t miss the hallmark listed in bold (the Job description). If it is for a sales/marketing/PR alike position, you can get away by overselling yourselves. For technical interview never promise things which you cannot deliver. I will let you a secret, the other side of the bench, also comprises of human beings, who themselves have done as many mistakes as you have done, if not more. They will be your future colleagues/supervisors. They reached their current position by committing those errors and are quite wary.
So what is the right measure? Simple tell them what you are good at, what you like and what you have done. It may not be a good idea to tell them what you don’t want, if you desperately need the job. In the long run however, you will find that the decent interviewer rejected you, not because you were bad but because you deserved better. I have been rejected roles in the final rounds with the comment from senior managers/leads saying there is an ‘aspiration mismatch’. Why hire a horse to do the donkeys’ work?

Presence of mind:

Listen, always listen first and ask for time if required. Ask sensible questions, If making a guess – spell it out loudly (I am not sure, can I guess!) It does wonders. There are dumb questions which get thrown to you, and smart answers which make the rational clear. Don’t jump the context unless you have struck a chord with the interviewer’s wavelength.
Interviewers also fumble; their wording is confusing at times, so ask questions so that you can think better. Do this before you admit defeat. If a interviewer is keen, hints will be thrown to you, counter points will be raised. This is your chance to take control. See someone is genuinely willing to hear you!

Honesty/Sincerity:

If above fails, admit your lack of knowledge or ignorance. It may be a morale breaker for you, but definitely not for the interview. People wish to judge on the merit and the intent. You won’t be punished for your lack of know how. No one can master every damn thing, but you definitely need to master the relevant. Even an indication that you are on the right course is a welcoming sight. There are only two things one has to do, communicate things which one has done and create trust about your ability to learn and adapt to new factors.

Never assume that you can drive the interview:

While it is true that most interviews have a defined timeline. It is never a good idea to gauge by the number of interviews and duration. The real metric is the quality of the hire. Technical folks tend to be adamant, so unless you give answers they don’t relent. It is never a good idea to procrastinate/show-off. Questions which ask you about your areas of comfort are to be viewed as the opening gambit. Sometimes pretty straight questions may come asking you to rate yourselves. The number is asked to judge ones perception of caliber and not as an answer. It is the interviewers’ job to do that. The best thing one can do is blend with the flow, like a sugar cube in milk, not obvious but definitely perceivable.
Interviewers also ask questions which sound dumb or outrageous; this is done just to set a context like a cold start.

1. 
   Think and communicate your thoughts/concerns
   


2. 
   Raise questions and point out flaws (politely)
   Wait for the control to be given to you, pounce by giving relevant examples from your experience.
   


3. 
   Always keep in mind that your preparation has started since the day you started working (or you joined college) and continual learning is indispensable.
   


4. 
   Never badmouth anyone. Infact this is one place where you can show-off you diplomatic skills.
   

Social Innovation - Communal participation (2)

In the earlier post, I listed out certain catalysts who have spent a significant time & effort in social causes. They are not to be seen as NGOs spending monies. These guys come under a different league. The inviting thing about social innovation, is that this can be copied/mimicked, and the plagiarist attitude is actually helpful. Anyone set up a franchise of a rural call center or courier service, and contribute in a meaningful way. I envy the HR in such organizations (near 0 attrition).

If deaf and dumb can run courier services, why not laundromats & cleaners? With 3G in India, why not have a call center that can transcribe and reinterpret the signs to verbal dialect targeted towards focus groups?. Why not come up with financial planning for rural farmers who have a different set of needs. Why not have a facebook for challenged/access deprived users? Can you have a broadcast station for deaf?

These ideas have great potential, and I for sure will not ask for any credit ;). Certain seemingly simple questions like transliteration of sign lang to text, have made me realize the complexity of the problem. Until then I would encourage and make an effort to participate in every effort that aids this cause.

Social Innovation - Communal participation (1)

Thanks to eenadu.net

Example 1

Who: Dhruv Lakra

What he did: Enabled the challenged folks to stand up for themselves

My moto for him: Teach a man how to fish...

Example 2

Who: Harish Hande

What he did: Brought sustainable power to masses

My moto for him: Your actions speak louder than your qualification.

Example 3

Who: Saloni malhotra

What she did: Proved that IT is not about A.C buildings

My moto for her: 'IT' makes sense

Example 4

Who: Ramana Babu Killi

What he did: Reached out to rural farmers

Maven and Ant

'Build' is more of a process than a tool. Build tools have come a long way, writing custom bat/shell scripts is no longer acceptable. Ant became the defacto standard for java shops. Then came Maven, for a developer working on a single project, the build tool would not be a major concern. It is just another tool in the arsenal.

A good Ant script is configurable, readable, modular and extensible. Properties and semantically sound naming conventions are must for a good script. This is analogous to writing maintainable code and carries the same penalty, none of the above conventions are mandatory, implying that hastily written scripts can end up in production environment.

Maven strongly enforces the concept of life-cycles. Plugins are used to carry out tasks (similar to ant tasks). Archetypes define the xml structure. All of these make it a bit more difficult to deviate from standards. Maven rules over Ant when it comes to standardisation. Almost anyone can use a maven build file (pom), where as certain build.xml files have known to have caused nightmares.

So why isn't everything mavenized? Answer: simplicity. Almost every developer working on a project know the libraries & versions required. Unless you are building a library/framework, one can be certain about build invariants (libraries). So 75.23455623% [random number generated by a fickle brain :)] of the projects meant for end-user consumption are happy with a working ant script. They don't need to haggle over dependency resolution. These are guys who don't care if SCM/CI are built-in or whether a GUI exists for viewing the jar files.

To conclude the discussion, decide in which environment you would be working. If it is a environment with diverse projects that may use your code one day, go with Maven as it is more polite to advanced users. If you work on personal projects/college assigments/maverick prototypes, Ant fits your bill.

Java HashMap with time base expiry policy

I recently had to come with this data-structure, later I found that google collections has a MapMaker which essentially does the same. Posting some ideas and the code I have written here.

There are broadly two different ideas:
1. Create a time tracker along with the hashMap. Then run a thread to detect and delete expired items
2. Create a new hashMap periodically while flushing the contents of old map.

I have implemented the first kind. This makes use of jdk 6 concurrency features

package test; import java.util.Collections; import java.util.HashMap; import java.util.HashSet; import java.util.Map; import java.util.Set; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.Executors; import java.util.concurrent.ScheduledExecutorService; import java.util.concurrent.ThreadFactory; import java.util.concurrent.TimeUnit; import java.util.concurrent.locks.ReentrantReadWriteLock; /** * Thread safe object map, which clears objects after a specified time. The objects are stored in the * underlying hashMap. * @author rongkan * */ public final class TimeLimitedCacheMap { private final ConcurrentHashMap<String, Object> objectMap = new ConcurrentHashMap<String, Object>(10); private final ConcurrentHashMap<String, Long> timeMap = new ConcurrentHashMap<String, Long>(); /* I need a shared lock, readwrite lock is an excellent candidate. * evcition is run with writeLock, put/remove with readLock */ private final ReentrantReadWriteLock accessLock = new ReentrantReadWriteLock(); //private final ReentrantLock accessLock = new ReentrantLock(); private final Runnable evictor = new Runnable() { /* evictor thread removes data, and changes map state. This is * in conflict with put() and remove(). So we need sync for these operations * * In case you are wondering why evictor needs sync (it just calls remove() right?) * eviction is a compound action that spans more than a single remove(). It enters * into a conflict with put() * * evictor: start looping ------------------------------> keyset is stale, evictor removes the recent put & armagedon comes * Thrd1:--------------------->put(same key, new object) * */ @Override public void run() { // avoid runs on empty maps if(timeMap.isEmpty()){ Thread.yield(); } long currentTime = System.nanoTime(); accessLock.writeLock().lock(); Set<String> keys = new HashSet<String>(timeMap.keySet()); accessLock.writeLock().unlock(); /* First attempt to detect & mark stale entries, but don't delete * The hash map may contain 1000s of objects dont' block it. The returned * Set returned may be stale, implying: * 1. contains keys for objects which are removed by user, using remove() (not a problem) * 2. contains keys for objects which are updated by user, using put() [a big problem] */ Set<String> markedForRemoval = new HashSet<String>(10); for (String key : keys) { long lastTime = timeMap.get(key); if(lastTime == 0){ continue; } long interval = currentTime - lastTime; long elapsedTime = TimeUnit.NANOSECONDS.convert(interval, expiryTimeUnit); if(elapsedTime > expiryTime){ markedForRemoval.add(key); } } /* Actual removal call, which runs on the objects marked earlier. * Assumption: marked objects.size() < hashmap.size() * Do not delete blindly, check for staleness before calling remove */ accessLock.writeLock().lock(); for (String key : markedForRemoval) { long lastTime = timeMap.get(key); if(lastTime == 0){ continue; } long interval = currentTime - lastTime; long elapsedTime = TimeUnit.NANOSECONDS.convert(interval, expiryTimeUnit); if(elapsedTime > expiryTime){ remove(key); } } accessLock.writeLock().unlock(); } }; private final ScheduledExecutorService timer = Executors.newSingleThreadScheduledExecutor(new MyThreadFactory(true)); private final class MyThreadFactory implements ThreadFactory { private boolean isDaemon = false; public MyThreadFactory(boolean daemon){ isDaemon = daemon; } @Override public Thread newThread(Runnable r) { Thread t = new Thread(r); t.setDaemon(isDaemon); return t; } }; private final long expiryTime; private final TimeUnit expiryTimeUnit; /** * Users can play around with evictionDelay and expiryTime. * 1. Large evictionDelay => less frequent checks, hence chances of finding expired Objects are more * 2. Lean evictionDelay => aggressive checks, and hence more sync overhead with put() and remove() * 3. Large expiryTime => increases the time object stays in object map and less chance of cache miss (cache miss is bad) * 4. Lean expiryTime => by itself does not force object to be removed aggressively, needs lean eviction to be configured * * In case you are wondering, above picture is not complete. * Another key aspect is "Arrival Periodicty", or the rate at which put() is called. * * Ideally: expiryTime == arrival periodicity + 1 [read '+1' as slightly greater] * evictionDelay == expiryTime + 1 * * For random arrival times, which is a more common scenario, use the following pointers * 1. eviction Delay > expiry Time * Here user needs to think of the impact of stale hit (define how stale is stale!) * 2. eviction Delay < arrival time * This has higher chances of cache miss and accidental treatment as failure * * 3. eviction Delay < expiry Time * Unwanted eviction run(s) resulting in sync overhead on map * 4. eviction Delay > arrival Time * Unwanted eviction run(s) resulting in sync overhead on map * * @param initialDelay, time after which scheduler starts * @param evictionDelay, periodicity with which eviction is carried out * @param expiryTime, age of the object, exceeding which the object is to be removed * @param unit */ public TimeLimitedCacheMap(long initialDelay, long evictionDelay, long expiryTime, TimeUnit unit){ timer.scheduleWithFixedDelay(evictor, initialDelay, evictionDelay, unit); this.expiryTime = expiryTime; this.expiryTimeUnit = unit; } /* The intention is to prevent user from modifying the object Map, * I want all adds/removals to be suspended. synchronizing on objectMap * would also work, but locks are easier to read without scope issues of {} * * Concurrent hashMap would have allowed put and remove to happen concurrently. * I did not use conc-map, because * 1. I want to update another map (timeMap) * 2. I want to sync the operation with evictor thread * * The unfortunate invariants: * 1. sync between evictor and put() * 2. sync between evictor and remove() * imply * 3. sync lock between put() and remove() * * 3. is unfortunate side effect, as you need to sync all exposed invariants on the same lock. * Lock duplication won't help. If I create putLock() and removeLock(), they will allow put() and remove() * to happen concurrently, but will not help two put()/remove() calls to happen in parallel. */ public void put(String key, Object value) { accessLock.readLock().lock(); Long nanoTime = System.nanoTime(); timeMap.put(key, nanoTime); objectMap.put(key, value); accessLock.readLock().unlock(); } /* Read comments for put() they apply here as well. * If had not allowed remove(), life would have been zillion times simpler. * However, an undoable action is quite bad. */ public Object remove(Object key) { accessLock.readLock().lock(); //accessLock.lock(); Object value = objectMap.remove(key); timeMap.remove(key); //accessLock.unlock(); accessLock.readLock().unlock(); return value; } /* Clone requires locking, to prevent the edge case where * the map is updated with clone is in progress */ public Map<String, Object> getClonedMap(){ accessLock.writeLock().lock(); HashMap<String, Object> mapClone = new HashMap<String, Object>(objectMap); accessLock.writeLock().unlock(); return Collections.unmodifiableMap(mapClone); } }

GXT Nuggets: Hipocrite's combo box

Bananas for the code monkey

Ever faced a situation where you want to show the user some information 'X' but you actually need 'Y' for the backend logic? Say you have an employee entity with a employee details and a auto-generated PK. You have designed a form to show the employee details, based on the user selection (in a drop-down combo). A hiprocrite's combo is up for this task. It shows employee name/number in the UI but secretly holds the PK deep down.

You may wonder why all the fuss, why not just set the state in the drop down. Works wonderfully provided you 'remember' the state's 'key'. If I have 10 such forms, this involves 10 if() stmts atleast. To overcome this and let the contract-driven design make life easier for us, I re-invented the combo box. I added an adaptive behaviour in form of DynamicProperyEditor. This enables us to query for various model properties without excessive intrusion.

public class DynamicPropertyEditor<Data extends BaseModelData> extends ListModelPropertyEditor<BaseModelData>{ private String propertyName; public DynamicPropertyEditor(String property) { this.propertyName = property; } public String getStringValue(Data value){ return value.get(propertyName); } public String getPropertyName() { return propertyName; } } public class X10ComboBox<D extends BaseModelData> extends ComboBox<D> { private DynamicPropertyEditor<D> dynamicPropertyEditor = null; public DynamicPropertyEditor<D> getDynamicPropertyEditor(){ return dynamicPropertyEditor; } public void setDynamicPropertyEditor(DynamicPropertyEditor<D> dynamicPropertyEditor) { this.dynamicPropertyEditor = dynamicPropertyEditor; } @Override public ListModelPropertyEditor<D> getPropertyEditor() { .... } }