The Dark Techie
Some random thoughts and feelings of The Dark Techie
Tuesday, May 10, 2016
Monday, March 14, 2016
My Best Friend,Teacher and Brother -Kingshuk
I had no idea even how to start and shutdown a computer when
I first entered the Computer Lab in our college in second semester. I was in
Computer Science major and like majority of the students in our class It was
expected that I would be knowing these basic of computer. But no, I didn’t
know. Unlike most of them I didn’t have a computer at home either. It was all a
big black box for me. I still don’t know whether I was supposed to know those
things while enrolling in college. But that is a different thing. I was shocked
to know that day that I had to learn computer to start learning computer at my
college.
I lost all interest and started bunking computer labs. This
made the situation even worse. I could not catchup on things, didn’t know what
is programming, how the loops work and how the hell the theory matches up with
practical.
The person who saved me during this time, is one of my best
friend forever was Kingshuk. He was the first person who taught me basically
what computer is, how can we have fun with computers how can we love it. He was
the first person who gave me set of software’s including “C” compilers and let
me play with them.
He gave me a CD full with softwares,flash animations,games and compilers. He also gave me a floppy drive.
Whenever i used to write any code I used to show to him. Even for silly codes also he always showed great encouragements.
Not only there, he stood by me till the last day of my college. When one by one all friends of mine got jobs and I couldnt make it, it was Kingshuk who always stood by me and supported.
Today whenever I have to write my hobby I write “Coding”. My
colleagues today know me as aboring geek.
But if I didn’t have Kingshuk that day by
my side, probably my Engineering would have ended with the frustration.
With time gradually lost touch with all my friends. Even with Kingshuk. We were best buddies. Modnapore,Kerani Tola, I promised many times but never visited.
I cannot reach him,dont have his number,contacts but still whenever I miss friend I miss him. Life changed ,got so many responsibilities and noticed oneday family is not enough and we need friend a good friend with whom we can talk rubbish for hours but I dont have any now.
Searching for him desperately.
Wednesday, March 25, 2015
Logwatcher!!! A Node Way to Visualize log event counts at Runtime
This application will keep count of hits and errors on an Apache HTTP server and will display both the counts on a single monitoring page. The counts are run-time, that means no more periodic polling on log file after certain interval rather the numbers on the page change as soon as the events occur on the server.
I was never convinced of polling a server after 5 minutes interval or 1 minute interval for checking hits/errors ,I always wanted these reports to be on the fly.
With the available resources and programming languages if we wanted to achieve the same we face several roadblocks, the most dangerous of locking the log file and also loading server with multiple threads.
Node blessed us with a very easy and convenient way of doing this by its Event Driven nature.
Apache HTTP server writes log / entry for each hit on server, to a log file. For my server it is access.log. This can be configured to use different file. Similarly for each error an entry is made in a log file error.log. Even this can be configured to use a different one.
Whenever I open a page , http://localhost/samplesite it counts as a hit on Apache and writes an entry in access.log. Even if I hit http://localhost that also does the same.
Similarly I have another PHP application which throws mysql error . If I open that application on browser by visiting http://localhost/errorApp then at the same time it will be a hit on Apache so access.log will get a new entry at the same time error.log also will get a new entry.
My monitoring page will get updated automatically with correct/updated number of total hits and errors as soon as a hit or error happens. you no need to re-fresh or no need to read/poll the log files periodically.
To avoid periodical polling on log file we should have a mechanism that will automatically trigger event when a file is modified.
fs.watchFile
This is the feature node provides that triggers an event whenever a watched file is changed. We can define what to do during that event in call back methods.
The syntax is "
fs.watchFile('./fileName.txt',callBack)
" Now whenever 'fileName.txt' file will be modified , the method 'callBack' will be invoked and executed.
How to update contents on all connected clients/browsers without refreshing?
socket.io
This is another feature that made communication over socket is real easy in Node. With socket IO, our node server will behave as server where as the served HTML pages will be each client and can send and receive communication to and from server through emitter.
So by now I hope the implementation logic has become very simple,
1. Watcher is attached on both the log files. Each call back will read linecount from respective log files and emit updated counts to all clients (i.e. to all who opened the monitoring page).
2. Upon receiving emitted data from Server the JS code on the HTML can update its content accordingly.
Below is my Node server file logWatcher.js
//Modules to load
var fs = require('fs');
var http = require('http');
var url = require('url');
var io = require('socket.io');
//Variables to declare
var accessFile='/var/log/apache2/access.log';
var errorFile='/var/log/apache2/error.log';
var serverPort=8080;
var count =0;
var countErr=0;
var all='';
var allError='';
var mTime='';
ErrmTime='';
var line = 0;
var initLine=0;
var errLine = 0;
var initError=0;
var server = http.createServer(function(request, response){
console.log("New Connection..");
var path = url.parse(request.url).pathname;
console.log(path);
switch(path){
case '/':
response.writeHead(200, {'Content-Type': 'text/html'});
response.write("Welcome to Apache Server Monitoring.
Click Here to Access The Monitoring Page");
break;
case '/monitor':
var data = fs.readFileSync('logView.html','utf8')
response.writeHead(200, {"Content-Type": "text/html"});
response.write(data, "utf8");
break;
default:
response.writeHead(404);
response.write('Gone case');
break;
}
response.end();
});
server.listen(serverPort);
console.log("Server Started..");
console.log("reading starts..");
var rstream = fs.createReadStream(accessFile);
rstream.on('data', function (chunk) {
line+= chunk.toString().split("\n").length;
})
.on('end', function () { // done
initLine=line;
});
console.log("Read ends..");
server.listen(serverPort);
console.log("Server Started..");
console.log("reading starts..");
var rstream = fs.createReadStream(errorFile);
rstream.on('data', function (chunk) {
errLine+= chunk.toString().split("\n").length;
})
.on('end', function () { // done
initError=errLine;
});
console.log("Read ends..");
var socket=io.listen(server); //attach socket on server
fs.watchFile(accessFile,callBackAccess);
fs.watchFile(errorFile,callBackError);
//Attach watcher on files on connection to display current status on new connections
socket.on('connection', function(socket){
console.log('....new connection..');
socket.emit('Update', {'message': all,'ErrMessage':allError,'Count':count,'Err':countErr});
});
////Commented code to tame fs.watchFile with readStream
function callBackAccess(cur,prev) {
console.log('hit..');
console.log("init line.."+initLine);
line=0;
var rstream = fs.createReadStream(accessFile);
rstream.on('data', function (chunk) {
line+= chunk.toString().split("\n").length;
})
.on('end', function () { // done
console.log("line...:"+line);
all=all+'
'+cur.mtime;
count=line-initLine;
socket.emit('Update', {'message': all,'ErrMessage':allError,'Count':count,'Err':countErr});
});
};
function callBackError(cur,prev) {
console.log("in error..");
errLine=0;
var rstream = fs.createReadStream(errorFile);
rstream.on('data', function (chunk) {
errLine+= chunk.toString().split("\n").length;
})
.on('end', function () { // done
countErr=errLine-initError;
});
allError=allError+'
'+cur.mtime;
socket.emit('Update', {'message': all,'ErrMessage':allError,'Count':count,'Err':countErr});
//ErrmTime=cur.mtime;
};
Below is source code for the logView.html
Theis page listens on the same socket for specific jason message with key "Update"
Upon recieving it updates the content of both the div tag with other details from that json message. At the same time it updates the title.
<html>
<head>
<script src="/socket.io/socket.io.js"></script>
</head>
<body>
<script>
var socket = io.connect();
socket.on('Update', function(data){
document.getElementById("msg").innerHTML=data.message;
document.getElementById("err").innerHTML=data.ErrMessage;
document.title=data.Count+':'+data.Err;
document.getElementById("count").innerHTML="Hit Count:"+data.Count;
document.getElementById("ErrCount").innerHTML="Error Count:"+data.Err;
});
</script>
<div id='count'></div><br>
<div id="msg"></div>
<hr>
<div id='ErrCount'></div><br>
<div id="err"></div>
</body>
</html>
Github repository for the complete and updated source codes :https://github.com/anisoftcorporation/logWatcher
Below are few screen shots of my logWatcher in action
![]() | |
| Starting the node application |
![]() |
| Access the App |
![]() | ||
| Monitoring Page at Initial State |
![]() | |
| Hiting Apache page. Note the hit count on title of monitoring page changes (in tab) |
![]() |
| Monitoring page after 1st hit on Apache |
![]() |
| Second Apache hit, another site on localhost , note hit count in other tab changed on the fly |
![]() |
| Accessing a site which causes error. Note the error count on the other tab got increased on the fly |
![]() |
| Opened another monitoring page and that new page also shows same information. See the first tab was the first monitoring page we opened and using so far |
![]() |
| After the node server is restarted all connected monitoring page gets refreshed automatically. The users never need to do refresh |
Friday, February 27, 2015
Yet another Node Vs PHP+Apache
When they said Node JS , they said about performance. They said about concurrency. They said about non-blocking I/O. I thought of giving a try to visualizing that in comparing with other languages and see if Node is really better in some way.
I will try here to explore the 2 main features of Node that make it win in certain scenarios.
1. Non-Blocking I/O: - In a typical application where we need to perform 3 tasks we simple execute them this way
task1();
task2();
task3();
Now task3() will not be executed until task2() is completed and task2() may take long time depending on its operation and so execution of task3() will be blocked. So in a scenario where all these jobs / tasks are independent to each other and can run in parallel, this sequential paradigm won't fit practically.
Node fits here perfectly. Node supports callbacks, that means calls are asynchronous and non-blocking. The task2() will not block task3() or in other words "a statement in Node will never wait for another I/O to be finished."
I created a similar scenario in Node
var http = require('http');
var resp = function doResp()
{
console.log("Second task executed..");
}
http.createServer(function (req, res) {
console.log("Task One exeuted.."); // This is task 1
setTimeout(resp,2000); //This is actually how callback is utilized
/* Below statements are task3*//
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end("Page content is loaded but request is yet processing task 2 in back ground...");
console.log("Task three executed..");
}).listen(9999);
The above simple node code has main 3 tasks to do as we planned.
Task one will print a message on console.
Task two is a Time consuming task takes 2 seconds and at the end it will display message on console.
Task three writes the response to user and also displays a message on console.
This line " setTimeout(resp,2000);" is where actually callback is used. So the actual function is being passed as parameter to setTimeout function , that means after 2000 millisecond the function doResp() will be invoked. Which is stored in a variable resp. But the beaut is the whole statement " setTimeout(resp,2000);" gets executed immediately and next statements start getting executed. Though in parallel sets a timer to execute the callback function after 2 seconds, it doesn't stop next tasks.
I started the app and hit from browser, and saw on Browser screen the message "
In console I could see the below 2 messages
"Task One exeuted.."
"Task three executed.."
"Task three executed.."
and after 2 seconds I did see the another message came up "Second task executed.."
I am happy with the Non-Blocking I/O (or to me asynchronous) of Node.
2. Concurrency: I am done with verifying nonblocking IO feature. Now I have to see and test concurrency and an overall load test. As suggested by others I used siege for load test simulation.
Wrote a simple Node which reads from a Mysql table and emmits JSON. Wrote another PHP file which does the same thing.
Here to make the response content length similar , I purposefully avoided Chunked transfer Encoding in PHP+Apache.
The table has around 53 records. I did test with few different parameters.
Complete code and database structure is available on Git link provided at the end.
Below are my complete test results. The upper ones are Node and Below are PHP
500 Concurrent users for 5 Minutes with 1 second delay
anirbanb2004@Anisoft-Corporation:~/www/loadTest/siegeLog$ siege -c500 -d1 -t5M -lnode1.log http://localhost:9615
** SIEGE 3.0.5
** Preparing 500 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 296509 hits
Availability: 100.00 %
Elapsed time: 299.75 secs
Data transferred: 830.79 MB
Response time: 0.00 secs
Transaction rate: 989.19 trans/sec
Throughput: 2.77 MB/sec
Concurrency: 4.20
Successful transactions: 296509
Failed transactions: 0
Longest transaction: 0.20
Shortest transaction: 0.00
FILE: node1.log
anirbanb2004@Anisoft-Corporation:~/www/loadTest/siegeLog$ siege -c500 -d1 -t5M -lnphp1.log http://localhost/loadTest/PHP/
** SIEGE 3.0.5
** Preparing 500 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 298656 hits
Availability: 100.00 %
Elapsed time: 299.92 secs
Data transferred: 836.80 MB
Response time: 0.00 secs
Transaction rate: 995.79 trans/sec
Throughput: 2.79 MB/sec
Concurrency: 2.27
Successful transactions: 298656
Failed transactions: 0
Longest transaction: 0.24
Shortest transaction: 0.00
FILE: nphp1.log
Summary : Not much difference. Concurrency is little higher and Longest transaction is also slightly better but nothing real impressive.
** SIEGE 3.0.5
** Preparing 500 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 296509 hits
Availability: 100.00 %
Elapsed time: 299.75 secs
Data transferred: 830.79 MB
Response time: 0.00 secs
Transaction rate: 989.19 trans/sec
Throughput: 2.77 MB/sec
Concurrency: 4.20
Successful transactions: 296509
Failed transactions: 0
Longest transaction: 0.20
Shortest transaction: 0.00
FILE: node1.log
anirbanb2004@Anisoft-Corporation:~/www/loadTest/siegeLog$ siege -c500 -d1 -t5M -lnphp1.log http://localhost/loadTest/PHP/
** SIEGE 3.0.5
** Preparing 500 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 298656 hits
Availability: 100.00 %
Elapsed time: 299.92 secs
Data transferred: 836.80 MB
Response time: 0.00 secs
Transaction rate: 995.79 trans/sec
Throughput: 2.79 MB/sec
Concurrency: 2.27
Successful transactions: 298656
Failed transactions: 0
Longest transaction: 0.24
Shortest transaction: 0.00
FILE: nphp1.log
Summary : Not much difference. Concurrency is little higher and Longest transaction is also slightly better but nothing real impressive.
500 Concurrent users for 10 minutes with 1 second delay
anirbanb2004@Anisoft-Corporation:~/www/loadTest/siegeLog$ siege -c500 -d1 -t10M -lnode1.log http://localhost:9615
** SIEGE 3.0.5
** Preparing 500 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 594697 hits
Availability: 100.00 %
Elapsed time: 599.95 secs
Data transferred: 1666.28 MB
Response time: 0.00 secs
Transaction rate: 991.24 trans/sec
Throughput: 2.78 MB/sec
Concurrency: 3.84
Successful transactions: 594697
Failed transactions: 0
Longest transaction: 0.20
Shortest transaction: 0.00
FILE: node1.log
anirbanb2004@Anisoft-Corporation:~/www/loadTest/siegeLog$ siege -c500 -d1 -t10M -lnphp1.log http://localhost/loadTest/PHP/
** SIEGE 3.0.5
** Preparing 500 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 597276 hits
Availability: 100.00 %
Elapsed time: 599.97 secs
Data transferred: 1673.50 MB
Response time: 0.00 secs
Transaction rate: 995.51 trans/sec
Throughput: 2.79 MB/sec
Concurrency: 1.94
Successful transactions: 597276
Failed transactions: 0
Longest transaction: 0.20
Shortest transaction: 0.00
FILE: nphp1.log
anirbanb2004@Anisoft-Corporation:~/www/loadTest/siegeLog$ siege -c500 -d1 -t10M -lnode1.log http://localhost:9615
** SIEGE 3.0.5
** Preparing 500 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 594697 hits
Availability: 100.00 %
Elapsed time: 599.95 secs
Data transferred: 1666.28 MB
Response time: 0.00 secs
Transaction rate: 991.24 trans/sec
Throughput: 2.78 MB/sec
Concurrency: 3.84
Successful transactions: 594697
Failed transactions: 0
Longest transaction: 0.20
Shortest transaction: 0.00
FILE: node1.log
anirbanb2004@Anisoft-Corporation:~/www/loadTest/siegeLog$ siege -c500 -d1 -t10M -lnphp1.log http://localhost/loadTest/PHP/
** SIEGE 3.0.5
** Preparing 500 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 597276 hits
Availability: 100.00 %
Elapsed time: 599.97 secs
Data transferred: 1673.50 MB
Response time: 0.00 secs
Transaction rate: 995.51 trans/sec
Throughput: 2.79 MB/sec
Concurrency: 1.94
Successful transactions: 597276
Failed transactions: 0
Longest transaction: 0.20
Shortest transaction: 0.00
FILE: nphp1.log
Summary: Node is still better at concurrency but yet nothing so impressive so far. I am still in love with my PHP+Apache
500 Concurrent Users for 15 minutes with 1 second delay
anirbanb2004@Anisoft-Corporation:~/www/loadTest/siegeLog$ siege -c500 -d1 -t15M -lnode1.log http://localhost:9615
** SIEGE 3.0.5
** Preparing 500 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 892874 hits
Availability: 100.00 %
Elapsed time: 899.25 secs
Data transferred: 2501.74 MB
Response time: 0.00 secs
Transaction rate: 992.91 trans/sec
Throughput: 2.78 MB/sec
Concurrency: 3.74
Successful transactions: 892874
Failed transactions: 0
Longest transaction: 0.25
Shortest transaction: 0.00
FILE: node1.log
anirbanb2004@Anisoft-Corporation:~/www/loadTest/siegeLog$ siege -c500 -d1 -t15M -lnphp1.log http://localhost/loadTest/PHP/
** SIEGE 3.0.5
** Preparing 500 concurrent users for battle.
The server is now under siege...
[error] socket: 1011885824 address is unavailable.: Cannot assign requested address
[error] socket: -213448960 address is unavailable.: Cannot assign requested address
[error] socket: -3631360 address is unavailable.: Cannot assign requested address
Lifting the server siege... done.
Transactions: 893948 hits
Availability: 99.96 %
Elapsed time: 899.70 secs
Data transferred: 2504.75 MB
Response time: 0.00 secs
Transaction rate: 993.61 trans/sec
Throughput: 2.78 MB/sec
Concurrency: 2.25
Successful transactions: 893948
Failed transactions: 335
Longest transaction: 1.01
Shortest transaction: 0.00
FILE: nphp1.log
anirbanb2004@Anisoft-Corporation:~/www/loadTest/siegeLog$ siege -c500 -d1 -t15M -lnode1.log http://localhost:9615
** SIEGE 3.0.5
** Preparing 500 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 892874 hits
Availability: 100.00 %
Elapsed time: 899.25 secs
Data transferred: 2501.74 MB
Response time: 0.00 secs
Transaction rate: 992.91 trans/sec
Throughput: 2.78 MB/sec
Concurrency: 3.74
Successful transactions: 892874
Failed transactions: 0
Longest transaction: 0.25
Shortest transaction: 0.00
FILE: node1.log
anirbanb2004@Anisoft-Corporation:~/www/loadTest/siegeLog$ siege -c500 -d1 -t15M -lnphp1.log http://localhost/loadTest/PHP/
** SIEGE 3.0.5
** Preparing 500 concurrent users for battle.
The server is now under siege...
[error] socket: 1011885824 address is unavailable.: Cannot assign requested address
[error] socket: -213448960 address is unavailable.: Cannot assign requested address
[error] socket: -3631360 address is unavailable.: Cannot assign requested address
Lifting the server siege... done.
Transactions: 893948 hits
Availability: 99.96 %
Elapsed time: 899.70 secs
Data transferred: 2504.75 MB
Response time: 0.00 secs
Transaction rate: 993.61 trans/sec
Throughput: 2.78 MB/sec
Concurrency: 2.25
Successful transactions: 893948
Failed transactions: 335
Longest transaction: 1.01
Shortest transaction: 0.00
FILE: nphp1.log
Summary: PHP breaks for these long running large number of concurrent requests.Server stopped creating threads when it created enough before max outing server's RAM. But node is standing straight here. Node looks even steady in terms of the parameters.
1K Concurrent users for 10 minutes with 1 Second delay
anirbanb2004@Anisoft-Corporation:~/www/loadTest/siegeLog$ siege -c1000 -d1 -t10M -lnode1.log http://localhost:9615
** SIEGE 3.0.5
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 862079 hits
Availability: 100.00 %
Elapsed time: 599.92 secs
Data transferred: 2415.46 MB
Response time: 0.20 secs
Transaction rate: 1436.99 trans/sec
Throughput: 4.03 MB/sec
Concurrency: 281.62
Successful transactions: 862079
Failed transactions: 0
Longest transaction: 0.55
Shortest transaction: 0.00
FILE: node1.log
anirbanb2004@Anisoft-Corporation:~/www/loadTest/siegeLog$ siege -c1000 -d1 -t10M -lnphp1.log http://localhost/loadTest/PHP/
** SIEGE 3.0.5
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 1193412 hits
Availability: 100.00 %
Elapsed time: 599.82 secs
Data transferred: 3343.82 MB
Response time: 0.00 secs
Transaction rate: 1989.62 trans/sec
Throughput: 5.57 MB/sec
Concurrency: 5.58
Successful transactions: 1193412
Failed transactions: 0
Longest transaction: 0.54
Shortest transaction: 0.00
FILE: nphp1.log
Summary: This shows the real advantage of node in terms of concurrency. See here with increase in number of concurrent users how the concurrency level increased for Node(281.62) where as for PHP+Apache it is 5.58 still.
anirbanb2004@Anisoft-Corporation:~/www/loadTest/siegeLog$ siege -c1000 -d1 -t10M -lnode1.log http://localhost:9615
** SIEGE 3.0.5
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 862079 hits
Availability: 100.00 %
Elapsed time: 599.92 secs
Data transferred: 2415.46 MB
Response time: 0.20 secs
Transaction rate: 1436.99 trans/sec
Throughput: 4.03 MB/sec
Concurrency: 281.62
Successful transactions: 862079
Failed transactions: 0
Longest transaction: 0.55
Shortest transaction: 0.00
FILE: node1.log
anirbanb2004@Anisoft-Corporation:~/www/loadTest/siegeLog$ siege -c1000 -d1 -t10M -lnphp1.log http://localhost/loadTest/PHP/
** SIEGE 3.0.5
** Preparing 1000 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 1193412 hits
Availability: 100.00 %
Elapsed time: 599.82 secs
Data transferred: 3343.82 MB
Response time: 0.00 secs
Transaction rate: 1989.62 trans/sec
Throughput: 5.57 MB/sec
Concurrency: 5.58
Successful transactions: 1193412
Failed transactions: 0
Longest transaction: 0.54
Shortest transaction: 0.00
FILE: nphp1.log
Summary: This shows the real advantage of node in terms of concurrency. See here with increase in number of concurrent users how the concurrency level increased for Node(281.62) where as for PHP+Apache it is 5.58 still.
Conclusion: So I am convinced from the above test that Node is good at when high very high concurrency is demanded. Also saw in first part that the nonblocking IO gives Node an edge for event based application.
At the same time realized that until unless a real huge concurrency is required no need for Node. The cost at-least at this moment will be higher for the below reason
- Getting Node experienced people or training your developer to Pro level is costlier
- PHP+Apache stack will consume more swap memory (RAM) when concurrency is more, but Node will be expensive on Processor (CPU) when performing CPU intensive jobs. Scaling up a server is cheaper when only dealing with memory or RAM (obviously within thresold) but if we need processor upgrade (on physical servers) that will be real tough and expensive job.
- If you are not owning dedicated server hosting Node on cloud is costlier than any other options
Someone in some forum wrote " You no need to bother and think about the concurrency of Node until you are managing sites like Google,Facebook or Amazon.
So it is upto Webmasters , depending on their load and traffic and also budget what they want to use and implement.
*Please note:i used mysql_() functions for ease of writing instead of using pdo or mysqli_() functions.
Tuesday, February 17, 2015
Late night reply to emails will not impress your Boss Anymore
There was a time when people used to work till very late night to show how hard he/she is working. Leaving office before boss leaves used to be considered as a serious crime :). People often do one more thing to show how much they are into the project and work by checking emails on their smart phones from home and respond quickly , even if it is midnight.
Those days are gone and gone those bosses as well. You cannot impress your boss by simply replying to email at midnight anymore. See below 5 reasons , why you should stay away from that trend.
1. Creating Dependency in Team: Team is built to support everyone. If you are responding to emails when you are not supposed to , your team members will be confused and will be expecting a solution from you. They will stop analyzing problems and always will await your mail first. This will not establish you as a Master Problem solver in team but it will only create a dependency on you in Team. No manager will love that and hence eventually your manager will not love you.
2. They will know you: Not only you, there are other people in team who love to be seen by others. When you are playing these tricks to impress your boss, those people will surely understand the game. Believe me , they will hate you. They will talk about you when you are not around. In one word your own team members won't love you.
3. You will make more mistakes: When you are responding to email at midnight , lying on your bed from a smart phone, you will hardly have patience to read through the mail completely, analyse properly and respond with proper information. I had a friend in one of the support project , who once responded to a mail at midnight by just saying "OK, I will do it". He didn't read the mail completely until next morning he woke up by a call from his manager. Now when he read the mail now and saw it was from a customer who was querying about delay in product arrival and was asking if some one can provide an update. It is easy to imagine how much that one liner response from him made the customer more frustrated. So your customer won't like you.
4. You are not giving 100% to family: I bet your wife,husband, kids won't love seeing you checking emails and responding them now and then. Never ever do the mistake of assuming that your family feel proud when they see you occupied always in your work. It is of pride for them to think or say you are very responsible person at work. At the same time it is frustrating for them to not having you with them when they do deserve. So , your family will not love you also.
5. Your boss is always smarter than you: The final and most important reason I feel , we all should remember is, however smart you are or you think you are, your boss is always smarter than you. Boss can understand every action you do and the reason behind it. By trying to fool your boss you are actually making fool of yourself only.
One cannot work or live with so many people disliking. So its better to take some actions to correct these.
There are times when working off hours or late night be required. These are emergencies but not routine. Nobody expects you to work 24 hours. If at any point you are feeling that there are so much work and 24 hours a day is even less for them to completed, stop for a moment and try to take the below steps.
a.) Speak to your boss about the situation
b.) Learn how to make realistic estimation
c.) Stop creating false expectation to others about your commitment and availability
d.) Learn how to delegate work effectively.
If you still cannot find personal space even after practicing the above steps, that simply means you are not suitable for the role you are assigned to.
Now don't blame your manager , if you don't get a better hike next time.
Monday, February 2, 2015
Email Verification - How To in PHP
While developing any web application, email verification is something that you must be doing as part of first step of the security for your web application. This feature confirms that real owner of an email address is actually using that email address and nobody else is creating account using the actual owner's email address.
This feature initially irritated the users who were used to click on signup button and were able to login instantly.
The extra actions of log in into mailboxes and searching for activation mail in inbox and then clicking on them, really bothered users those days. But this adds an extra step of security for identity threat. Now you cannot find a website where you won't be needing email or phone number validation. In this blog I will discuss step by step how can the email address verification be added to a PHP based web application.
Step 1: Database (Split it up)
For dynamic web applications database does play a very crucial role because most of the contents of the application are stored in database and they get pulled up logically based on the code and logic written on the pages.
All user's information are stored in database tables. A lazy and early days programmer store all user information in one table. But to implement the email verification feature and for the sake of best practices we will segregate user's attributes into 2 categories and use 2 separate tables
This table will have regular attributes for login except the below 2
1. act_key
2. status
The 'status' column will have integer value and we will consider
0 = Activation Required
1 = Active
The act_key will hold the encrypted value of the Activation key or OTP.
So the complete user_account_master will look as below
Step 2: Signup Page
This is a simple HTML page with a very simple form. This page serves as the first page where users will be asked to provide only few details required for signup.
User Name
Password
Confirm Password
Email Address
Confirm Email Address
So our basic Signup.html will have the below HTML code
<form action="register.php" method="POST">
Username <input type="text" name="uname"/><br>
Email Address <input type="text" name="email"/><br>
Password <input type="password" name="passwd"/><br>
Retype Password <input type="password" name="c_password"/><br>
<input type="submit" name="submit" value="Register"/><br>
Step 3 : register.php
Once the previous HTML page is submitted the form will be posted to a PHP page. In this PHP page will do the below actions one by one
rand(1,100)
Step 1: Database (Split it up)
For dynamic web applications database does play a very crucial role because most of the contents of the application are stored in database and they get pulled up logically based on the code and logic written on the pages.
All user's information are stored in database tables. A lazy and early days programmer store all user information in one table. But to implement the email verification feature and for the sake of best practices we will segregate user's attributes into 2 categories and use 2 separate tables
- user_account_master - For Account Attributes
- user_profile_master - For Profile Attributes
Account Attributes are attributes necessary to have users' account set up and ready to authenticate.
Examples of Account attributes are , username, email address, password etc.
Profile Attributes are the additional attributes related to the users. Examples of profile attributes are First name, Last name, Address etc.
In this post we will deal with only account attributes so the table we will be considering is only user_account_master.Examples of Account attributes are , username, email address, password etc.
Profile Attributes are the additional attributes related to the users. Examples of profile attributes are First name, Last name, Address etc.
This table will have regular attributes for login except the below 2
1. act_key
2. status
The 'status' column will have integer value and we will consider
0 = Activation Required
1 = Active
The act_key will hold the encrypted value of the Activation key or OTP.
So the complete user_account_master will look as below
![]() |
| user_account_master |
Step 2: Signup Page
This is a simple HTML page with a very simple form. This page serves as the first page where users will be asked to provide only few details required for signup.
User Name
Password
Confirm Password
Email Address
Confirm Email Address
So our basic Signup.html will have the below HTML code
<form action="register.php" method="POST">
Username <input type="text" name="uname"/><br>
Email Address <input type="text" name="email"/><br>
Password <input type="password" name="passwd"/><br>
Retype Password <input type="password" name="c_password"/><br>
<input type="submit" name="submit" value="Register"/><br>
</form>
Step 3 : register.php
Once the previous HTML page is submitted the form will be posted to a PHP page. In this PHP page will do the below actions one by one
- Generate a random key.
rand(1,100)
- Hash that key (I used MD5). I used below code to generate and hash altogether.
- Insert other account details and the hashed key into database. While inserting the value of status column will be 0
$uname = $_REQUEST['uname'];
$password=$_REQUEST['passwd'];
$email=$_REQUEST['email'];
$key = md5(rand(1,100));
$query="insert into user_account_master values('','$uname','$email','$password',now(),0,'$key')";
mysql_query($query);
$password=$_REQUEST['passwd'];
$email=$_REQUEST['email'];
$key = md5(rand(1,100));
$query="insert into user_account_master values('','$uname','$email','$password',now(),0,'$key')";
mysql_query($query);
- Generate the content of the email and activation url to be sent to user.
Query the user_account_master table to get UID for the newly created user. While querying can search with email or user-name as those are unique. Then form a url to activate.php with UID and Key in header as below.
http://example.com/activation.php?uid=12345&key=xxxxxxxxxxxxxxxxxxx
I used a function to generate the mail content and the code for the funstion is as below
function generateMailBody($uname,$uid,$key)
{
$body =CONTENT1.$uname.CONTENT2.CONTENT3.CONTENT4.CONTENT5.SITE_URL."/activate.php?uid=".$uid."&key=".$key.CONTENT6.CONTENT7.CONTENT8.SITE_NAME;
return $body;
{
$body =CONTENT1.$uname.CONTENT2.CONTENT3.CONTENT4.CONTENT5.SITE_URL."/activate.php?uid=".$uid."&key=".$key.CONTENT6.CONTENT7.CONTENT8.SITE_NAME;
return $body;
}
The mail content will look like below
![]() |
| Activation Email Content |
Where clicking on the link will take user to the url
http://localhost/signup/phpMailVerification/activate.php?uid=15&key=e4da3b7fbbce2345d7772b0674a318d5
The database entry for newly created user at this stage will look like below
![]() |
| After Signup |
Step 4: Activation Code - activate.php
In this example the activate.php will serve the action of activating account upon receiving exact act_key for specific user. This page will take 2 request parameters
http://example.com/activate.php?uid=12345&key=xxxxxxxxxxxxxxxxxxx
This page will simply update the status in user_account_master table for the record matching the uid and key passed in url.
$uid=$_REQUEST['uid'];
$key=$_REQUEST['key'];
- uid
- key
http://example.com/activate.php?uid=12345&key=xxxxxxxxxxxxxxxxxxx
This page will simply update the status in user_account_master table for the record matching the uid and key passed in url.
$uid=$_REQUEST['uid'];
$key=$_REQUEST['key'];
$update_query ="update user_account_master set state=1 where uid=$uid and act_key='$key'";
mysql_query($update_query);
$num= mysql_affected_rows();
mysql_query($update_query);
$num= mysql_affected_rows();
Now the $num will be 1if any record is updated else it will be 0.
So if we get $num=1 we can display a success message else a failure message.
After successful activation the same entry in database will look like below
![]() |
| After Activation |
5. Secure Login Page
The login page is the final part where we need to put a check for user status. While querying data-base for successful login we must query with username , password and status.
username and password will be passed by users and status must be equals to 1 to be able to login.
This way it will restrict users from login until unless email account is verified.
Simple login html will look as below
![]() |
| Login Form |
The code for login will be as below
$uname=$_REQUEST['uname'];
$password=$_REQUEST['passwd'];
$login_query="select * from user_account_master where uname='$uname' and password='$password' and state=1";
$login_result=mysql_query($login_query);
$num_rows=mysql_num_rows($login_result);
$password=$_REQUEST['passwd'];
$login_query="select * from user_account_master where uname='$uname' and password='$password' and state=1";
$login_result=mysql_query($login_query);
$num_rows=mysql_num_rows($login_result);
If $num_rows =1 then login is successful , else login will be considered failure.
For the 2 users in database screen-shot below login will be successful for test91 , while it will fail for test92 as email is not yet verified and account is not yet activated (state=0)
![]() |
| Both types of accounts |
Final words
This is a basic tutorial to show how to do basic email verification / account activation. This can be leveraged to develop features like OTP , forgot password etc.
All the source codes used for this blog and the MySql table structure are on the below Github workspace https://github.com/anisoftcorporation/phpMailVerification.git
I welcome readers' comments and suggestions.
Tuesday, January 13, 2015
Problem Solving and You
BElieve it or not IT Service industry is basically problem driven. Most of the jobs and projects are either for solving existing problems or to safeguard for future problems.
There are projects where the task is to develop a new product but even in that also the majority of the effort is spent on solving problems or in planning to stop possible problems.
There are projects where the task is to develop a new product but even in that also the majority of the effort is spent on solving problems or in planning to stop possible problems.
If we look around in real life , we will see in everyday life also most of the tasks we do are either solving some problem or planning to prevent future problems.
So no doubt problem solving is a great quality to have and almost all of us have this quality.
But the way we solve problems differ from one person to another. The way of solving problems or looking at problem actually comes from within our characters and if we can identify someone's way of handling problems we can actually assess that person's nature.
So no doubt problem solving is a great quality to have and almost all of us have this quality.
But the way we solve problems differ from one person to another. The way of solving problems or looking at problem actually comes from within our characters and if we can identify someone's way of handling problems we can actually assess that person's nature.
There are naturally two types of problem solvers around us.
1. Quick Fixer
2. Root finder
1. Quick Fixer
2. Root finder
I am not considering other type of people here who prepy cannot solve problems and run away from it. We will talk about those two types I mentioned above.
Quick Fixer
These type of people work on problems immediately to fix it. When a problem arrives the one and only aim they have to make the system up and running again. Think of a situation when server of an Enterprise goes down all of a sudden. The target of the quick fixer will be to bring it up as soon as possible. And when it is up and running the quick fixer will be satisfied.
These type of people work on problems immediately to fix it. When a problem arrives the one and only aim they have to make the system up and running again. Think of a situation when server of an Enterprise goes down all of a sudden. The target of the quick fixer will be to bring it up as soon as possible. And when it is up and running the quick fixer will be satisfied.
Root Finder
These types of problem solvers emphasize firstly on finding the root cause of the problem than fixing the current problem. In the above scenario of server crash in the enterprise this problem solver will look for the reason that caused the server crash and then will fix that root cause rather.
These types of problem solvers emphasize firstly on finding the root cause of the problem than fixing the current problem. In the above scenario of server crash in the enterprise this problem solver will look for the reason that caused the server crash and then will fix that root cause rather.
So Who Is Better?
It is tricky and conditional. Though seeing description of the both and their examples readers may think Type 2 is the correct one that a problem solver should be but this answer is not so easy.
It is tricky and conditional. Though seeing description of the both and their examples readers may think Type 2 is the correct one that a problem solver should be but this answer is not so easy.
What if the person needs 2 days to identify the root cause? Will the server of the Enterprise be down and unavailable for these whole 2 days? Can any corporate afford such prolonged business outage? No. Nobody will or can.
They will definitely be needing a Quick Fixer at first place. But does that mean there is no need for Root Finder? No. They do need to find the root cause definitely. Otherwise there might be outage again and that may cause severe impact on customers faith in their business.
They will definitely be needing a Quick Fixer at first place. But does that mean there is no need for Root Finder? No. They do need to find the root cause definitely. Otherwise there might be outage again and that may cause severe impact on customers faith in their business.
So bottom line is both types of problem solvers are required and they both have their own value.
Now the most strange statement I am going to say "A person cannot be both by heart for a same problem". Yes you read it correct. Though in modern practice all IT Support people will do the fix first and then will be engaged in a root cause analysis. But in most of the cases both are not done by same person. If done by same person then either of the two will not be done from that person's heart.
Now the most strange statement I am going to say "A person cannot be both by heart for a same problem". Yes you read it correct. Though in modern practice all IT Support people will do the fix first and then will be engaged in a root cause analysis. But in most of the cases both are not done by same person. If done by same person then either of the two will not be done from that person's heart.
Think a root finder is in charge of support of infra structure of a big customer. Now for any outage that person needs to make sure that the system is up and running as soon as possible. But being a root finder it will not be possible for that person to find a quick fix to the problem. That person is meant to think analytically and will come up with root cause but that will impact business. So to solve this different kinds of documentations and support run books are prepared. The person follows them to make the system available as soon as possible. But this is not spontaneous.
These are not limited to IT industry. It is the same in all industries and in in fact our daily lives. We see tasks and problems in our lives. We deal with them different ways. Some solves problems as and when they come and never think about those again. Some think about the problems and solve it at root. Like IT in lives also both types of solution required. But the difference is like in corporate we don't have multiple solvers to solve our own personal problems. So depending on the type of problems we need to act. For simple example if we feel hungry or tired we cannot sit and try to figure out root cause rather we solve by eating or taking rest. If you see your monthly savings is decreasing then you have nothing to do when you notice rather than identifying the extra expenses and planning on how to save more in future.
Behavioral Traits
As I said at beginning by noticing the way of problem solving we can identify certain nature of the person. I will explain that here now. These are purely based on my observation and may differ.
As I said at beginning by noticing the way of problem solving we can identify certain nature of the person. I will explain that here now. These are purely based on my observation and may differ.
Root Finder
- A root finder is basically composed and introvert persons.
- They don't speak much. Another nature is they don't act on impulse.
- As a nature they are little lazy and love in taking time on solving problems
- They want to be sure before speaking of anything
- They love reading
Quick Fixer
- Quick Fixers are in opposite hand little more expressive
- They are more talkative and work on events
- They are more dynamic and ready to act in no time
- They do what they feel at the moment
- They hate reading
As a conclusion human behavior cannot be defined so easily. These are what I have seen most but are not bible to judge human nature. You have to do your own way. But certainly while solving that problem you can choose what type of solver you want to be and share in the comment.
Have a great day.
Subscribe to:
Posts (Atom)

















