Logging Pitfall: Let’s just log everything! What could go wrong?


So last week I overheard one of our developers’ discussion with the account manager who happens to be a tech person as well. They were talking about errors encountered by a client’s website written in Python Django. The developer reported that the site just began crashing whenever this or that script starts to write things to the website’s log file and they were expecting that the user under which the site runs has not enough permission to perform logging. It was reasonable since the whole website was moved from one hosting provider to another and it might have been encountering configuration issues. I ignored it. I was preparing for a prototype presentation that afternoon.

That same developer works for my team and even joined me in the presentation. When we got back to the office, she said that she would have to stay a little longer to work on the issue she was discussing with the account manager earlier. That’s when I offered my help. It was Friday afternoon. Although I still got a lot in my task list, I knew that I better start on them the following week.

I asked her to describe to me how the site runs. She said that the site has this main script that runs all the other scripts. The other scripts perform the logging. I asked her to show me a sample of the logs and she showed me what was on her screen. I was expecting a log meant for audit trails but the logs appeared to be those default logs for debugging the website. I could have argued that it should no longer be there in production but I let it go. She was doing maintenance work anyway and debug logs can be helpful. I then asked her if everything was working before. She said yes. I told her it should have nothing to do with permissions then unless the hosting provider messed up with user permissions in the server. She seemed a bit puzzled of what I was trying to point out.

Log files are developers’ best friends. When I was still a student and was developing using Windows’ classic notepad and was not yet acquainted to debugging tools, I referred to logs for errors and “watched” variables. I tended to log everything when frustrated. It was helping me; what could go wrong? I left snippets of code meant to log things when I submitted my works just so I can have a ready reference once something goes wrong. Only when I started working did I realize the problem it could introduce if not done properly.

Programming languages and frameworks are designed to perform things fast. Show a tech person a website that takes a long time to load in the browser for whatever reason be it internet speed, architecture, etc. and somebody will raise one of their eyebrows. This same capability of frameworks to perform processes fast makes a lot of developers forget to properly handle logging.

Concurrency. This can be the number one problem if logging is mishandled. And the annoying thing about it is that you know as a developer that you should be wary about concurrency issues but you missed this one about concurrent log file access because you were treating logging as a small thing. And to make it worse, your website will behave as expected as long as one script finishes logging to the file before another starts writing its own logs; something that is very possible until the website gets large traffic.

I told our developer that Python is a very fast language and that’s probably why the website did okay for years with all the logging that was happening. But now that it’s handling a lot of requests, it’s now encountering concurrent file access issues. For a quick test and patch I suggested that she use different log files and see if the website will still encounter the issue. She reported that it worked. I asked her if it was really necessary to put everything in a single log file but she said she would have to confirm with the client first. I then just told her that if it’s really necessary, she should create a handler that stores all logs in memory and then flushes it in the file after about one to three seconds. There will be a delay but no logs should be lost and there should no more of the IO exceptions that the site is currently encountering. I will find out later. -aB

Featured Image by: ~no-reaction

The UPCAT 2014 Results Out Now


See online results.

Go here for searchable results.

The results of the University of the Philippines College Admission Test (UPCAT) 2014 is now out. Congratulations to all those whose hard labor has been fruitful and have passed one of the most challenging entrance examinations in the country. This is just the beginning of the challenge, we all know that, but for now celebrate for your triumph. I wish you all the best. Have fun on your “toxic” days.

Also, shout out to the new batch of BS Computer Science students from one of your kuyas.

Student Number Name Campus Degree Program
2014-30700 AGIR, KHARL GAEBRIEL AGUILAR MANILA BS COMPUTER SCIENCE
2014-10559 AGUZAR, STEPHANIE ANNE BUTIAL MANILA BS COMPUTER SCIENCE
2014-27736 ALANO, ANGELIKA FAYE SERRANO MANILA BS COMPUTER SCIENCE
2014-25817 ANGELES, SIGFREED JOHN SALCEDO MANILA BS COMPUTER SCIENCE
2014-19252 BARCELONA, DIANE BOLANTE MANILA BS COMPUTER SCIENCE
2014-47783 BUNAG, KENRICK LANCE TAN MANILA BS COMPUTER SCIENCE
2014-50177 CAPARAS, ALETHEA MARI MANILA BS COMPUTER SCIENCE
2014-07077 CHUA, JANE KIMBERLY NIU MANILA BS COMPUTER SCIENCE
2014-49148 CHUATECO, MARIKA DELOS SANTOS MANILA BS COMPUTER SCIENCE
2014-05834 DE JESUS, ANIKA PAULA MATSUO MANILA BS COMPUTER SCIENCE
2014-43376 ESPALLARDO, LLOYD VINCENT ANDARZA MANILA BS COMPUTER SCIENCE
2014-01882 FLORENDO, PRINCESS DANIELLE VALDEZ MANILA BS COMPUTER SCIENCE
2014-55403 GAN, PATRICK LAWRENCE CUA MANILA BS COMPUTER SCIENCE
2014-04108 LAGUDA, REECE ADRIAN CORTES MANILA BS COMPUTER SCIENCE
2014-26004 LIM, JOURDAN RILEY LIM MANILA BS COMPUTER SCIENCE
2014-27438 LIPATA, ANTHONY SOLATORIO MANILA BS COMPUTER SCIENCE
2014-41075 MARPA, DARLENE PSALM GONZALES MANILA BS COMPUTER SCIENCE
2014-00308 MENDOZA, DELWYN PERLAS MANILA BS COMPUTER SCIENCE
2014-41079 MORA, GRACE IRENE ZAMBRANO MANILA BS COMPUTER SCIENCE
2014-40979 OCAMPO, RAFAEL ANGELO SALVADOR MANILA BS COMPUTER SCIENCE
2014-01842 ONGCHAN, HANSSEN PIERRE TAN MANILA BS COMPUTER SCIENCE
2014-32819 ONGCHINKE, JAYSON DELA PENA MANILA BS COMPUTER SCIENCE
2014-61400 PEREZ, JOSHUA ANDREW BELARMINO MANILA BS COMPUTER SCIENCE
2014-56730 QUITO, RAI EARL DON SAN JOSE MANILA BS COMPUTER SCIENCE
2014-05888 REGILME, CHARLENE GRACE SAN JOSE MANILA BS COMPUTER SCIENCE
2014-10019 ROMERO, ROBERT EMMANUEL REYES MANILA BS COMPUTER SCIENCE
2014-60201 SAMONTE, CHERRY ANN PANGANIBAN MANILA BS COMPUTER SCIENCE
2014-66980 SILMARO, BIANCA CAMILLE LAGUE MANILA BS COMPUTER SCIENCE
2014-10868 TABORNAL, NEIL KENNETH CAMACHO MANILA BS COMPUTER SCIENCE
2014-20458 TEJERERO, EDGAR ANTHONY LOYOLA MANILA BS COMPUTER SCIENCE
2014-54778 UY, JOHN BENGEMIN SIH MANILA BS COMPUTER SCIENCE
2014-00731 VILLALUNA, WINFRED LOUIE DELA CRUZ MANILA BS COMPUTER SCIENCE
2014-01740 VIRTUS, ANGEL PATRICIA RUIZ MANILA BS COMPUTER SCIENCE
2014-23703 YAP, RICHELLE SAN MARTIN MANILA BS COMPUTER SCIENCE
2014-02360 ZAMORA, ABIGAIL GALGANA MANILA BS COMPUTER SCIENCE
2014-64917 ZAMUDIO, GABRIEL PORCALLA MANILA BS COMPUTER SCIENCE

Have You Tried Turning It Off Then On Again?


arvinbadiola:

I found myself seeking better challenges in the past few months. And yes, I moved out of the company after just three months. This time, I went for what I want than what’s convenient.

Originally posted on theradioman:

I just moved. Perhaps to a lot of people’s dismay except mine, I moved out of a convenient place and joined a startup. Thanks to my habit of keeping everybody’s contact information, I was able to get back to an opportunity I previously declined by sending in a carefully thought email at 1:10 in the morning.

The idea had been playing around in my head for quite a long while. I knew that I wanted it and that it was possible but I was having second thoughts. It was not about conformity in a country where most job seekers go for the big names that obviously offer big compensation and benefits; I know how to get along well with people but I’ve never really seen myself as a conformist. It was that fear of committing a bad life changing decision. It’s always easy to know what we want but we…

View original 448 more words

Creating a Ribbon using HTML and CSS


A couple of weeks ago, I have stumbled upon this tutorial on how to create arrows using plain CSS & HTML and was kind of dumbfounded. I realized maybe we turn a little too much into Photoshop and other drawing tools these days that we forget the things we can actually accomplish with CSS & HTML alone. Of course one can argue that it’s easier to create arrow images but let us set that aside for now.

I thought of creating something similar; a ribbon using plain CSS & HTML. No not the bow. I’m pertaining to the ribbon we usually use as bookmark symbol, or an indicator of something we can click or pull down in our websites.

Below is my HTML code. I simply named the top part of our ribbon, “ribbon-head”, and the snake tongue-looking bottom part as the “ribbon-tail”.

<div id="ribbon-head">
 <h1>Ribbn</h1>
</div>
<div id="ribbon-tail">
 <div id="left"></div>
 <div id="right"></div>
</div>

Below is my CSS code.


/* head */
#ribbon-head { display: block; background-color: red; width: 100px; height: 100px; }

/* text */
h1 { font-family: 'Impact'; color: #fff; text-align: center; padding-top: 33%; }

/* tail */
#ribbon-tail { display: table-row; }
#left, #right { display: table-cell; border-top: 50px solid red; border-bottom: 50px solid transparent; height: 0px; width: 0px; }
#left {  border-left: 50px solid red; }
#right { border-right: 50px solid red; }

The codes shall give you something like the ribbon below.

ribbn

I honestly haven't thought about this a lot so feel free to comment your ideas on how this can be improved. -aB

I Want a Side Job


arvinbadiola:

Yes… I’m currently looking for a side job.

Originally posted on theradioman:

I was carefully putting on my shoes when I told my mom the news. I’ve just put two plasters over an open wound on my right foot and I was avoiding friction. “I am actually looking for a part time job.” I wasn’t looking but I have expected a blank stare. “Why would you look for another job?” she shot back. Alright, maybe I was expecting something a little too dramatic or Hollywood.

It was six in the morning and I was about to leave for work. I could have stopped right there and saved myself from a lengthy conversation. I knew she would disapprove. She couldn’t see the point and the tendency that I, her son, could go from slightly underweight to skin and bones worries her like every other mom I know. “Why would you do such a thing? You don’t have a family that you need to…

View original 344 more words

“The Issuance of NBI Clearances could make Use of Automation” thank God this Post is no longer important


I would have posted this a long time ago and I was thankful I didn’t have to. NBI now has a solution. It may not be excellent but it works for now. I can say it’s already a step to getting things working better for all of us. But I’m presenting this anyway just to share the experience and as something we could all look back into. Enjoy reading!

I went to our municipal hall one Monday to get an NBI clearance. I am moving to another company and it’s one of the pre-employment requirements. The office won’t be open until eight in the morning so I went there at around 7:30 AM. I was advised that the wait will be awfully long so I decided to go earlier than I would normally prefer. My adviser couldn’t have been any more right.

There were already a number of people in the line when I got there but I didn’t mind. I just wanted to get things done early but not necessarily be the first one to do so. The office opened at 8 AM which earned them credit for being on time but the rest of the experience wasn’t very flattering. At 10:30 AM I have just finished Step 4 which was encoding and verification of your personal information. That was already two and a half hours and there are six steps.

For those who are not aware, here are steps 1 to 6:

  1. Line up for the application form.
  2. Fill out the form and line up for data validation/verification by one of their personnels.
  3. Line up for payment.
  4. Line up for data encoding. Make sure everything you wrote on the form was encoded right.
  5. Line up for fingerprinting and photo capture.
  6. Wait for release of your clearance. If your receipt was marked a different date, you can leave and just claim your clearance on the said date.

I had to put “line up” several times; I find it the best way to describe things to readers. There was too much of it that it will probably be the first thing you would recall about the experience.

Now let us shift things a bit, automation of office processes particularly registrations or applications is a common task to IT people. In fact it’s too common it’s already boring even for students. I could still remember that for the same reason, I chose to go for implementation of artificial neural networks for my dissertation; a relatively harder task. Being someone unsure of how they will fare in the IT industry, I wasn’t particularly a challenge-seeker. I simply wanted to do something different; something that was not boring. But no matter how boring it could be, the benefit of having processes automated for both the organization and the people they deal with is undeniable, it gives everybody more time for other things. If you still find it hard to realize that benefit, try buying yourself two and a half hours to add to your day.

The issuance of NBI clearances could make use of automation. The process is simple enough that I believe even volunteer programmers could do it. This is not to say that volunteer programmers are not good analysts. This is to say that the task is too simple that programmers could choose to do it for free and take it as volunteer work. Just imagine how faster things could be if they will choose to automate even just half of the registration process.

Contrary to what you might assume, I’m not the the type of person who wants almost everything automated. In fact although it’s a bit shameful for me to admit, I could say that I’m still a pen and paper person. But in this case of processing applications for NBI clearances, I just find it very reasonable to employ automation. The single-paged paper-based application form is very easy to create perhaps in an HTML form and the validation performed by the municipal hall personnels is similarly easy to implement both in client side and server side scripts. Also, the user will be mostly in charge of verifying correctness of their personal information, a task they could accomplish in a minute or so. Having just that, an online form, applicants can already be saved from the two-hour-long wait and use of paper can be massively reduced. Because if one would think about it, writing down things on a piece of paper just to be typed in by someone else to a computer is actually ridiculous to begin with. You need not be an excellent typer to do something like that. Well the forms can already be downloaded online but it doesn’t help that much if you would still need to line up for someone to validate the information you wrote down on it.

Processing payment is a different thing for not a lot of applicants possess credit cards or Paypal accounts, but programmers could certainly put it as a nice option. But since you would still have to appear to the office to get your picture and fingerprints taken, maybe you can just hand your payment there or present a printed receipt.

A lot of software developers will still frown at this solution for its lack of elegance. Certainly the whole process could be automated. But I’m merely stating what can be readily accomplished in a couple of days (maybe a day for some) and this post is not meant to discuss a solution architecture. The point is that the application process could be better without exerting too much effort so why isn’t there somebody working on it?

The rest of that Monday was uneventful so I don’t really have the right to rant about the application process wasting my time but then not everybody has that much time to spare. We could make things better. Let’s do it now. -aB

A Beautiful Story from Teehan+Lax


So I was going through my emails this morning when I came upon my Twitter suggestions for this week. I usually just quickly read through these suggestions as I am not very active in Twitter. I simply use my account to disseminate information I gathered somewhere in the web and original content I post through my blogs.

But I digress, here’s what I really got to share. Teehan+Lax‘s twitter account was one of the suggested accounts this morning and having read their description that they do UX design, I got curious and visited their website. (I don’t search a lot of information on the hottest companies so please forgive my ignorance.) Needless to say, they got everything you would expect from a company specializing on user experience. But what really got to me was their story which you can read over here.

Learnings from Teehan+Lax

The things they’ve learned.

I won’t make this a very long one as their story’s already pretty long; but it was a good and very inspiring read for everybody. It also brings in the humanity in how they present their company to people, unlike others who are pretty high hat on how they do it.

If you’re going to ask me if they are paying me for this in any way, NO. Some stories are just worth sharing especially if there’s a lot that you could learn from them. Cheers for everybody’s happy future! -aB