Friday, February 08, 2008
The TCL Vision - Where will we be by 2020?
Thursday, September 13, 2007
Jon Wright goes to Boston
PEST Boston III – September 5th 2007
Yesterday was PEST Boston III, my first PEST!
I arrived in Boston the day before – my first time in America – and although passport control and customs were a bit of a trial (secondary passport control established I knew who I was and who I work for, and the sniffer dog established that I sometimes carry sandwiches or fruit to work in my laptop bag!) I eventually collected my rental car – with satnav – and drove straight into the first tunnel out of Boston Logan International…
Since then I’ve been having a brilliant time – among others, I’ve met with people from Northeastern University and Wellesley College, and tomorrow I’m meeting with Stanley Sclaroff of Boston University, where we are about to commence a new scholarship program – exciting times!
It has been really great to meet some people I had already heard about from previous PESTs, and there were a couple of new faces in the group who everyone enjoyed meeting – Hongwei Xi from Boston University, and Sara-Jane Haven from Tufts Health Plan.
The venue was Owen O’Leary’s in Southborough, which worked really well for the number of people we had – but the suggestion from PEST II of a round table in a more secluded area still stands, and if anyone has any suggestions please let us know!
We had two experience reports in the evening – the first from Nancy O’Leary of HP, on automation, which led to an interesting discussion that could easily have gone on for ages had Don not gently reminded me of my role for the evening J
The second experience report came from Dan Downing of Mentora – a case study on the subject of load and performance testing – a thought-provoking account of working with an organisation who appeared to have made some classic mistakes.
Things we learnt this time:
- Automation tools should have comprehensive evalutaion versions to assess their usefulness.
- Beer can have bits of fruit in it and still taste good (our server was pushing the Plum Beer to get a free T-shirt, which we managed for her!)
- Estimation of the maximum expected number of concurrent users for an application can be calculated in some interesting ways.
- Throwing hardware at a performance problem can easily make things worse not better overall.
- Architecture problems might be obvious, but if they're hard to solve they may stay an 'elephant in the room' until it's too late.
- Academia and industry see some things in different ways and each can learn from the other - it was good to have Hongwei with us at this PEST to give a slightly different perspective!
- It is important to establish a set of metrics for automation tools, so their efficiency can be accurately determined for repeat usage
Our PEST website is nearly ready to go live, so hopefully we will soon have a forum for the continuation of some of the lively discussions we have had, and a place for people to keep in touch between events.
In the meantime if anyone would like more information about PEST (or about our scholarship programs) please send me or Stewart an email.
I hope to meet everyone again at some point in the future – thank you to everyone who attended for making it such a good evening. Remember the point of PEST: none of us is as smart as all of us!
Jon (jonathan.wright@transitionconsulting.co.uk)
P.S. Additional thanks to Don St. Pierre for his helpful advice and for booking the venue!
Monday, August 06, 2007
PEST III - 05 September 2007
The venue is yet to be decided, but likely to be around the Marlborough area again, as this worked out well for most people that came along last time.
Steve Govednik (monster.com) and Dan Downing(Mentora) have voluntered for giving experience reports/challenges to the group next time.
Nadereh (mathworks) and Nancy (HP) have agreed to do stuff for the session after/ be alternates for the session in September.
If you want to come along please send me a mail to confirm, and also please look to bring a buddy. We'd like to get the attendance above 20 this time if possible :)
Stew
PEST Boston II - 18 July 2007
So here we are in Boston, and its PEST II at the Naked Fish restaurant.
Everyone who came last time was here, and we had some new friends too in the form of colleagues from HP Bangalore and Monstor.com.
I was very excited about coming to this PEST. Partly because I was looking forward to seeing everyone again - last time was a lot of fun, and partly because I wanted to tell everyone about our new scholarship scheme which is just starting up at Boston University. Our first outside of the UK, which will start in September this year, and have interns with us in summer '08. Its a very personal passion of mine to invest in the future of our industry, and its wonderful to see the scholarship schemes starting all over the world.
Although we still haven't cracked the venue side of things - it was a bit noisy even in this separated out area - everyone contributed well and we had some great discussions right from the start.
With such a mix of hardware/software experience along with the international flavours from the background of those attending US (7 people), India (2), UK (1), Iranian (1) we had some amazingly different view points as well as a lot of synergy.
The session started with an experience report from our own Don St Pierre, who was taking part in PEST as his last official act with TCL USA. He's off to join EMC from next week, and it was great timing that he could do this before he left.
Dons experience report focused mainly around the experiences he documented in his white paper last October - about the need for separation between QA and development reporting lines, and we were lucky to have Steve with us who had worked with Don on the same project and could add his ideas and recollections too.
We drew on these experiences to look at how we report from testing, and particularly how a set of metrics such as DRE can be used to create an 'apples with apples' comparison across projects. Across the group there was also a lot of experience of how metrics and measurements can hurt you as much as help - with the 'what gets measured gets done' approach that we see a lot with teams.
We also recognised that you can generate 'behaviours that exactly meet the metrics, rather than make sense' - things like if you measure a team on numbers of bugs raised then you get every little typo raised as a separate entry and lots of time wasted.
Now as you can see I remembered my camera so we have a couple of pictures which is great!
(Watch out for this lady at the front of the table - shes a trouble maker ;) )
Don did some great prep work for this session and gave out copies of his white paper and some definitions of DRE. We also shared some best practices from TCL in terms of using the DRE metrics at gateways and to help create measurement of process maturity.
Some other things that we learnt between us across the session included:
- There is a job website called Dice which costs only $60 per posting, compared to Monster.com which is circa $360
- QA can often be mistakenly seen as the root cause of defects
- Root cause analysis on defects is very important in a continuous improvement environment. How can we set up our defect tracking to then generate suitable reports to do this? Perhaps a topic for next time.
- Common performance issues are missed when environments do not represent live in a quantifiable / scalable way
- Upper management can often try to micro manage QA - and when they do they look at things they understand rather than (perhaps) the things that are actually important
- Date and budget driven objectives that don't include quality - and particularly post live quality metrics generate very unhelpful behaviours on projects
- Bringing customer found bugs into the post implementation review and post live defect tracking is very important in a continuous learning environment
- Release specific scorecards are very useful - particularly when they track DRE
- Public vs Private metrics - what can you track and what can/should you publish. A difficult balance
- Reporting to senior managers, VPs and Presidents means finding things that both interest them and help us - you cant just talk quality, you need to talk value.
That's about all for now, except to say:
- I'm hoping to have a private room, with a round table for the next PEST Boston - probably at the naked fish restaurant again
- Steve Govednik (monster.com) and Dan Downing(Mentora) have volunteered for giving experience reports/challenges to the group next time
- Nadereh (mathworks) and Nancy (HP) have agreed to do stuff for the session after/ be alternates for the session in September
- I'm going to look to bring some of the scholarship essays along for the group to have a look at - some of them have been really brilliant
- Don will be coming along to the next PEST, and has promised Matt hes going to bring a few people from EMC too!
Stew
The point of PEST: None of us is as smart as all of us!
Tuesday, June 26, 2007
PEST Boston- Metrowest July 18th at the Naked Fish in Westborough Ma.
The PEST will be at the Naked Fish in Westborough, MA, on the WEST bound side of Route 9. The new date is Wednesday, July 18th starting at about 5:30 pm. The subject for this PEST will be how to improve defect removal efficiency and approaches used.
I have a more private area, a room where we should be able to talk better.
Thanks and we look forward to seeing you!
Thursday, June 14, 2007
PEST "BOSTON" in MARLBOROUGH
PEST Boston - May ‘07
So here I am in Boston and it’s a beautiful day. Matt and I have just been to see HP, at their amazing campus offices in Marlborough – No I have no idea if this is where the cigarettes come from! – and now its time to get ready for PEST Boston!
This is the first one we have run in the USA, and unfortunately after sending out the invites and getting the tables booked at a local Pub 99 Don St Pierre has had a bit of an aberrant week and can"t make it along and so its me and Matt that are going to entertain and excite people about testing!
Luckily, we have a good crowd – or who knows how it might have ended up !
Great to see our friend from Mentora, Dan Downing who came along and got involved straight away. His stories from similar peer sharing forums really opened up the forum and there were some instant lively discussions.
Chris Reeves from Sepaton, Nancy O’Leary from HP, Mike Raia from Gtech and Nadereh Rooein from Mathworks all joined us for the session and got seriously stuck into the sharing from the start. Some tall, cold beers set the scene and half a dozen platters of appetizers also helped matters along.
Our theme today: How to improve the efficiency and effectiveness of testing. A hard one, as it has so much scope, so we used experience reports from two projects that I worked on to create a focus. We talked about how the supplier relationships, process and controls were used along with the questions we had to ask ourselves and the vendors about demonstrating test effectiveness.
The conversation bubbled and boiled into different areas of thought and experience from across our diverse backgrounds which included a great deal of knowledge about hardware and performance testing – two of my very weakest areas. We also got into the tricky area of how to shape and prioritise testing and of course this led to a session on the basics of Risk Based Testing.
2.5 hours well spent. Our ‘AA’ style of ‘Hi, my name is Stew and I’m a tester!’ (followed by a resounding: Hi Stew !) really worked and everyone was like old friends by the time we finished up.
‘So’, I hear you ask! What did you come out with, other than a nice bar tab?
Well:
That testing problems in the UK (such as deadlines and code that doesn’t work and environments that don’t support a basic smoke test etc) are the same in the US
That Risk Based Testing really can help shape things
Not all testers think about the business, or the value of the product they are helping to bring to market
Static testing really can bring down the cost of the overall project
Offshore resourcing doesn’t always get a positive response in the US
There really are 60,000 ways to present statistics including defect detection profiles that are normalised for tester hours e.g. defect number per tester hour
PEST really is a good format
When you are in charge of marketing you really should take a camera with you everywhere so that you can take pictures of things like groups of people at the first PEST Boston.
That beer and good conversation go together well.
We agreed to do it again next quarter, and I think everyone who attended this time will be back. They are all going to look around the office for a buddy to bring with them. That should make for a bigger group – and we’ll have to think about where we hold the next session as the Pub 99 won't cope well with many more people than we had this time (noisy, hard to always hear).
Most rewarding of all for me from this session is that people went back to work and started talking with their teams about Risk Based Testing, and looking at how to make it work for them. Now that is a result!
Over Q2 there will be PESTS run in:
London – Tony Prosser (Tony.Prosser@TransitionConsulting.co.uk)
Bristol – Richard Morgan (Richard.Morgan@TransitionConsulting.co.uk)
Boston – Don St Pierre (Don.St.Pierre@TCL.US.COM)
Bangalore – Grant Obermaier. (Grant.Obermaier@TransitionConsulting.co.uk)
If you want to get involved with any of these then please speak with the organiser to get a place reserved. I think the theme in Q2 will be: How to improve Defect Removal Efficiency, however it is pretty much the way of these things that anything goes and experience – and particularly experience reports – count for everything. If you have an idea, or a war story to share, then don’t hesitate to get involved. Who knows, you might help someone to do stuff better. You might even learning something yourself!
Remember the point of PEST: None of us is as smart as all of us.
Stew
Friday, February 09, 2007
Since I attended the class and passed the ASTQB Certified Tester, Foundation Level, I received several emails asking me what ASTQB is.ASTQB was founded in 2003 as the American Testing Board.
In April 2005, the name was changed to the American Software Testing Qualifications Board. This program has been developed by some of the world's foremost experts in software testing. The program is intended for software professionals who want to prepare for today's testing challenges as well as anticipating tomorrow's software testing demands. This program has been developed in cooperation with the International Software Testing Qualifications Board (ISTQB), organized by ASQF and International Software Quality Institute (ISQI), introducing a truly international standard for software. Cool! Now I am certified world wide!!
Ok why am I telling you this? TCL-US will be offering the class at a substantial discount. The class will run from March 7th through March 9th. If interested please send email to don.st.pierre@tcl.us.com. The class will be held in our corporate headquarters in Burlington, Mass. The class will be delivered by a seasoned trainer who has trained many certified testers and has a high pass rate. Don't delay; this class will fill up quick
