Effort vs Efficiency

[Steven Sim Kok Leong]

Hi,

What now? Another of Steven Sim Kok Leong's nonsense? :P

I have been observing Bill Hassell's scores for quite some time. It definitely does not go unnoticed that the number of bunnies he has is a lot more than many forumers (including me) that have accumulated more points than him (likely because many of us spent more time posting on the forums than him). With the number of pages of responses he has accumulated, it is without doubt that he is a very efficient contributor.

It is of little doubt both effort and efficiency ultimately achieves the same goal of effectiveness in the forums. The current ranking is based on the cumulative effects of both effort and efficiency.

I am wondering if a ranking based on just efficiency alone would be nice to have. I would define efficiency as the ability to answer a user's query to the best of the user's satisfaction (bunny!).

1) A close estimate would be the quotient of:

the number of bunnies / the number of responses

2) A rough estimate would be:

the number of pages of bunnies / the number of pages of responses

There are some deviating factors I can notice to this quotient:
1) not all users give points
2) not all users follow the scale to give points
3) a forumer might look for easy questions to answer just to gain points
4) there must be a lower bound of postings to gauge accurage. Someone who posts a single solution and gets a 10/10 doesn't mean he is the most efficient forumer.

However, by counting a large number of postings or pages, such biased factors might possibly be diminished to a considerable extent.

Just my 2 cents of very humble opinion. I am simply curious to hear your opinion of:

1) Whether this would be a good metric to have?

My opinion is that this doesn't add obvious immediate advantages to the forums. But in the long run, with both existing rankings and perhaps efficiency rankings, forumers may come up actively with both more answers (current state) as well as increasing more accurate ones (improving current accuracy).

2) Whether this be a fair-enough metric to measure efficiency?

This would be considering the deviating factors and perhaps others that I have missed.

I am also curious to know whether Bill Hassell or someone else is the most efficient contributor in the forums.

Disclaimer: I don't mean to create unnecessary work and I don't mean to displease anyone *sincerely*.

Just seeking some opinions I (and perhaps others) might just be curious to find out.

Just simply my humble 2 cents of opinion. Please feel free to comment. Regards.

Steven Sim Kok Leong

[Gnananandhan]

Hi Steve,
I think,The user who posts the question will be the best judge than other methods. Because he is the one who is experimenting it.
If someone give more points for a simple question - It means he is so innocent in that particular environment or denotes his criticalness or importance to the problem.
People who answers only for simple questions - We should welcome them also, They are sharing their level of knowledge.


Regards,
Gnana A.

[Paula J Frazer-Campbell]

Steven

I see your point but to "put the cat among the pigeons" (stir things up).

Are you saying that for example a Wizard with 1000 points gained with 200 x 5 point answers should not be ranked the same as another Wizard with 1000 points gained with 125 x 8 point answers.


;^)


Paula

[Carlos Fernandez Riera]

He Stenven:

Bill is a master... that's all.


The art of writing is not easy, even if you are writing in your native tongue. If you write a doc and try to read it again you will feel that it was not exatly what you want to say. People whit this skill take advantage. James is a clear example; Minimum effort max. efficiency.

I think the color of your hat ( if any) is another factor for newcomers ( but it is still discused).

In the other hand, when you read lots of replys you realize that everyone write on his/her way, just aiming the cause or resolution or showing a full solution.


No comments on rankings... :-)).

[Bill McNAMARA_1]

This is possibly true for the hpux forums, however, other forums such as the storage/ workstations and servers have a very low points assignment rate.
Note also that replies to questions that you authored are also included.. Bill Hassell unsuprisingly only posted a few questions.
Added to this multiple replies to the same question also show up.

I think the best criteria to efficiency is points per day of membership to the forums.

Later,
Bill

[Justo Exposito]

Hi Steven,

Perhaps a "Forumer of the month hat" is posible and this mark a person that is most efficient in this month.

??what about this?

I'm with Paula, Points are Points.

Regards,

Justo.

[Nick Wickens]

Oooh I don't like Bills idea (Nick Wickens 316 points & forum member since may 06 1997 !!).

Remember that most members join the forum as I did in the early stages of learning HP-UX and it takes a while to be confident enough in your abilities to contribute.

I would question whether any additional changes to the system would be of any great benefit. After all the aim of the forums is self help for HP users and while the points system is fun, does it really need to be taken too seriously.

If anyone wants to know who gets the most bunny to points ratio then they can simply look at the stats produced occasionally by procura.

[Bill McNAMARA_1]

I agree with you Nick!

Later,
Bill

[Pete Randall]

Steven,

It's an interesting idea but frankly, it's just another statistic. One of the problems on the forums is our obsession with ranking ourselves, the race for points, etc. I don't think creating another metric is really going to accomplish much other than give us point mongers another way to compare each other.

For What It's Worth,
Pete

[Devbinder Singh Marway]

I believe efficiency comes with experience. The longer and more exposed you become in your area of expertise the more efficent you will be. I believe this can be a bit dangerous , because users who post a question may request for a particular user to answer their question , simply because they have more rabbits than anyone else .

[Tim D Fulford]

Remember that the forum points system is also used as a guide for searches. Whilst initially a thread may answere one users specific question, in the future it may help 10 people with similar problems and no postings. In fact some of my collegues only use this forum as a search & make no submissions.

I have not figured a way, but it might be useful if there were some way to attribute points (this may just be internal points & not "bunny" points), say, on the number of times a specific query was looked up & did the search successfully help the user.

Just an Idea, my 0.02??? worth

Tim

[H.Merijn Brand (procura]

Chasing this ...

Might be in the next statistics (only for members that have more points than the previous fetch)

[John Bolene]

I do like the procura rating listing (which HP should adopt, since it is hard to tell past the top 25).

I don't see any reason or need for an efficiency award.

It is hard enough just to get people to assign points.

And we really are not in this forum to care much about the points, it is more about the answers. The points are just gravy and eye candy to make the experience more rewarding.

My 2 cents.

[Stefan Farrelly]

Bill should be given a +50,000 point advantage over the rest of us just for experience, in my humble opinion.

This point thing is far too subjective. You answer well lots of questions some days, get no points, answer easy questions some days and get lots. In my opinion the only way to make this more accurate if the most experienced HP person around (namely Bill) reviewed answers by others and sort of 'marked' them to put them in order - just like a test.

Ive seen a creeping habbit lately of people with huge points totals but cleary are not experienced as their answers are very poor advice most of the time. I hate to think how they are accumulating points.

[Stefan Farrelly]

Oh, I forgot to add. All of us NOT in the USA region are at a severe disadvantage over accumulating points anyway as by far the largest number of questions comes up during the US daytime, which is end of day/night in the UK, so those outside the US should get a point adjustment anyway in the region of 100% because while we're awake we have so much fewer possible questions to answer!

[Martin Johnson]

I like Bill's suggestion. I joined the forums on May 8th, 2002. I currently have 346 points. Not bad considering that 2 weeks of that time I was on vacation and didn't go near a computer.


Marty

[Mark Greene_1]

Steven,

As a stats freak, I can appreciate where you are coming from. As a realist, I think that this is too complex to be implemented, and then would only have limited utility.

The biggest showstopper that I see is trying to do any statistical smoothing based on the complexity of the questions asked and the answers provided. This is because some very simply phrased questions may have very complex, or even complicated answers; and, some of the more complex questions that go into great lengths and detail to describe the given situation may have a completely simple answer (e.g. kill HUP the named pid).

You are quite correct in your observation that not everyone scores the same. My biggest frustration with the scoring is not the people who don't give points, but rather those who award 5 or less to the correct answers because it's not what they want to hear. For all the bunnies that Bill and Clay and the rest of the olympians and pharoahs have, I've seen them given a 5 or less for what was indeed the correct solution on too many occasions to count.

This condition is inherant to the human species. If you find a way to effectively deal with that, by all means publish it and then wait to hear from the Nobel Prize committee!

Seriously, I think the best option is for Procura to incorporate some of these into his monthly stats effort. Of course, given the issues he as with gather some of the information in order to produce the stats, this may not be really doable at all.

HTH
mark

[Nick Wickens]

.. To add to Stephans point - thanks to BT most of us in the UK also have connect with 56k modems or ISDN if we are lucky so I think we should get more points simply for patience :@).

P.S If I have given any bad advice then just blame my first HP-UX teacher - The guy with the blue and yellow head gear just above :@).

[Stefan Farrelly]

Hey Nick, that couldnt have been me could it ? surely you'd been on an HP course before we met ?? At least you havent miraculously accumulated several thousand points already:-)

Drop me a line; stefan_farrelly@ipcmedia.com

[H.Merijn Brand (procura]

Mark, it can be done, I've done it :)

But it needs more than avarage bandwidth to make the story complete, as with the bunny detection which needs the same hoopjumps.

I rather have the 'm'/'j' problem solved than these futile wishes, so I can find you in the list too. And I want the country back.

I agree with all that the value of a grade like this is arguable, but for the statistic freaks is is just another interesting number, and for me it is just another temptation to try if it is doable.

Once again, in my statistics there are errors, and there is not a single personal preference in why the current ordering is done. Paula posted the first statistics, and there were a few foloow-ups. I just could not resist the temptation of finding out how it could be done without mirroring the complete ITRC

Once the j and m members are in again, I want you all to do some checks and give feedback of how the statistics could be improved. Not that they are useful, but they seem to satisfy some questions, and who am I to not give information that is rather easy to put together (once you know how to let perl do what it's good at)


Now for a personal opinion: I agree with Stefan, that we in Europe have a big disadvantage over US members foir exactly the reasons he states

[A. Clay Stephenson]

I think this is a rather meaningless number. The data is badly skewed. Without naming names, I can cite two recent threads (yesterday) which illustrate the point. In one, the person received 4 accurate answers within minutes and the highest points awarded was 4. The person was even asked if the answers were somehow wrong. In another thread (the dreaded touchy-feely type), the two forum members who happen to wear green salad got 10 points each and the host of other equally valid answers got zip. I have seen many cases where I have received many more points for an answer when there were better answers provided by a person with a lesser or even no hat.

The other side of this issue of 'effort vs. efficiency' is how much is doing a search worth? I place very little value in those responses and that is something the questioner should have been able to do for himself/herself.

I'm done with my rant now. I hate this touchy-feely stuff.


Regards, Clay

[Steven Sim Kok Leong]

Hi all,

Thanks for your wonderful responses. :-) Because opinions are subjective, I have given everyone the same points.

Ghana, my opinion is that you are right that efficiency should not have any relationship to do with the degree of a question, as long as the question is answered to full user satisfaction.

Paula, you seem to understand my lingo best. Yes, I was wondering if an efficiency measure would get ourselves improve on the accuracy in our answers. Yes, I am saying that under efficiency, a Wizard with 1000 points gained with 125 x 8 point answers should be ranked higher than a Wizard with 1000 points gained with 200 x 5 point answers. After reading the responses, perhaps the word ranking alone does not sound healthy afterall. Rephrasing it on a heathier note, understanding my own % efficiency might be helpful in improving the accuracy of responses in the forums.

Carlos, yes, my personal desire would be to be growingly more efficient and effective in my responses.

Bill, point noted: the deviating factor of a different scale of assignment in other forums would render this quotient unreliable and thus ineffective in its cause. Points per day of membership: there is one agree and one disagree on that in following responses.

Justo, yes points are just points though there are many facets to it. Previous threads have discussed that. And as Clay puts it, "touchy-feely" stuff. The concern with the deviating factors seem to be high (based on the responses) so it would appear that forumer of the month based on efficiency may not go well with everyone.

Nick, yes I was discussing with Procura along similar lines in the chatroom before I posted this posting. From a statistical viewpoint, which is more accurate i.e. No. of bunnies / total points or No. of bunnies / total responses?

Pete, I was thinking from the the perspective of the forum itself that efficiency might probably help in improving the accuracy of postings. I would personally think that having an efficiency number would meant that the issue of race for points which could result in less accurate answers (i.e. mass-accumulation of many small points) could be mitigated.

Devbinder, so far I have not seen that occurring often (at least not for me), unless it is a follow-up question to a specific responder's response.

Tim, the question of FAQ. I think this was raised the last time in the forum regarding having the most-asked questions and the best combined answer to be listed. I personally think that a search which prioritises such FAQ or technical documents before forum answers might be more efficent, because technical documents are usually more complete.

Procura, give you something to practise your perl on. You and your pe(a)?rls ;-)

John, I agree on the Procura part. However, points are still important to be able to tell how accurate a solution is. And the search that depends on this would benefit those seeking answers, making it more efficient in their searches.

Stefan, noted: the subjectiveness would render the quotient unreliable.

Mark, noted: the complexity would rander the quotient inaccurate in depicting the actual scenerio. :) Yeap, had discussed a while with Procura before sending out this post to solicit opinions.

Procura, its your call, as long as it doesn't overload the forums. :)

Clay, thanks for your opinion on this touchy-feely issue. :) Noted: this statistic is worthless because it is subjective because statistic is based on the person who posted the question.

Thanks again for your opinion. I hope I summarised them correctly. Afterall, interpreting the sentences and wordings is again subjective. ;-) I hope this had been a useful poll.

Conclusion: the general consensus is that this statistic is not worthwhile (also not for the ultimate goal of improving the accuracy of forum responses) except for statistic freaks :P

Regards.

Steven Sim Kok Leong

[James R. Ferguson]

Hi Gang:

I've been following this thread with interest since its inception. Steven, as always, you have offered another tantalizing morsel.

Obviously, the object of points is to denote value or usefulness. We have long agreed that points are the 'bread crumbs' by which we can find the best answers to our questions. Thus, it follows that refining the ability to measure value should lead to an improvement in finding solutions, either directly or indirectly by inference.

Heisenberg postulated that "The more precisely the position is determined, the less precisely the momentum is known in this instant, and vice versa." Philosophically this is can be translated to say, that the harder we try to measure something, the more we perturb the very thing we are trying to measure.

Personally, I enjoy the statistics that Wodisch and Merry (Procura) offer, and would like to see them continue "unofficially". Personally, like you and others, I want to sharpen my ability to offer better answers and any vehicle that guides me in that direction is welcome.

Anecdotally, Clay points out a wonderful example of just how poorly the point system can measure responses. Incidentally, in one of the two threads he cited, the author did, return to award meager points to all the "non-salad-hats". Incidentally, in the second thread mentioned, that author noted in his closure that there had been additional information that he had neglected to provide.

Stefan has lamented, at least twice recently, that sometimes points have been awarded for wrong or misleading replies that in the worst case could cause downtime or data loss. In another thread, he stated that, "Too many people lately have gotten new hats and this in no way means they're experienced enough to offer good advice, just the opposite..."

I would offer, again, that as participating professionals, we have an obligation to politely correct or challenge bad or dangerous advice when and as we see it.

The point system exists not only to provide the pointer to value, but also to reward those who volunteer their time. If you want more points, volunteer more often. As Paula noted elsewhere, "If you help then hats will come".

I like the point system for the satisfaction it provides me when I have helped. I certainly like it as a pointer to (hopefully) the better data, and I enjoy the statistics that Wodisch and Merry offer us. I personally use points and the kind of statistics suggested, as a ruler to measure and enhance my contributions. Use all of this as your own guide, in your own way, for your own purpose.

I think the "point" is to keep points in perspective.

With my regards to all!

...JRF...

[Jeff Schussele]

Steven,

You're a good man & obviously an execellent SA. But I have to agree w/Clay. The exercise is fundamentally futile. Simply *because* the point assignment is up to the whim of the questioner. There can NEVER be a consistent way to judge responders ability... based solely upon point assignment. NO MATTER how you slice & dice it there are FAR too many variables involved.

And frankly..I'm stepping out on that limb now , hoping it won't crack...I'm somewhat disappointed with, what I perceive as whining from some of the Euro gang about recent point gains from some of us Yanks. Just because we dwell in the States. HEY, we didn't force you to live in Europe. YES we obviously have an advantage...hey numbers are numbers. But that's simply because of where we live. IF you truly think the "quality" of the answers we Yanks give is seriously less that what was given in the past...well I'll leave that up to the forum veterans to decide. I wasn't here & I have no way to know & will decline to speculate. Besides, it's always been a "buyers beware" situation, I don't think that's ever changed.

Could it be that you veteran Euro forumers "scared" earlier Yanks away? Get off your damn high horse & just contribute...for no other reason than that's the fricking heart of this forum...ISN'T IT?

All I know is that I answer questions. I answer simply because I have the desire to try to help people...AND TO LEARN. I know that there are superior SAs out there. But are you to say that simply because of that I shouldn't contribute? Hell, if you've got a problem with my response, WHY aren't you countering it with the "proper" advice? What...if you say it's "crap" where are you? If you know better, why aren't you stepping up?

JRF, Clay, Harry....I don't care, if I think they're "wrong" or missing something - I say it!
I've done it before & I'll do it in the future.
And if I'm wrong I fully expect them to tell me. And if they convince me they're right & I'm wrong, well damnit, I'LL ADMIT it - like a man & then thank them for educating me.

Isn't that, in the big view, what we're here for? Let's try to lessen the acrimony here & remember what the FUNDAMENTAL reason WHY this forum exists....KNOWLEDGE TRANSFER.

I for one don't give a damn about points. I simply have them because I answer questions...maybe more than others...maybe more than you. But if because as a Yank I "get" to answer more than you...well if that upsets you, I really think that's your problem. And it's not my fault & I owe you nothing. Nor will I change my forum habits simply to accomodate you.

OK, my rant's over...I just want to say that I truly respect & appreciate ALL you forumers. You're all SUPER people & damn fine professionals in my book. Let's just try to keep the personalities out of the forum...except for senses of humor. For Christ's sake, we are human beings!!!

My 2-1/2 cents,
Jeff

[Wodisch]

Hi Steven,

like you and the others already confirmed, every kind of a point-system has a lot of weakness built-in :-(
The number of "bunnies" (males and/or females ;-) are as unreliable as the mere points - all they show is dedication to answer...

"procura" has got a nice idea about showing the delta in points (his *fast-mover* statistics), as not all of us answer at a constant rate!
Take myself as example (I guess I count as an "veteran Euro forumer" and maybe even as an "human being"):
- sometimes I am able to hang out on the forums (learning AND answering) for hours per day,
- sometimes I am *not connected* at all (and hence "silent")
so the mere points would always show that I am "someone", but procura's delta-statistics would not even know me sometimes!
That "delta" on a monthly base seem to be "good enough" for me (and fair enoug, too)!

Oh, and some answers were quite simple to me, so it took only seconds to answer (about the time to write), but some need testing and "research", maybe hours (but I am willing to spend them, if the question seems to be one that I might encounter myself, or believe I should know the answer, or such). "Efficiency" would measure how much of *my* time it takes to deliver the answer AND how much time it takes for the asking person to understand/verify/check/use... difficult to measure, at least! So, someone like Bill Hassel might need only minutes to deliver answers, where it could take hours for us "mere mortals".

So, no, I do not think it's a good/fair metric in absolute terms - but the best we can get (or simply "good enough").

Oh, and I do NOT believe that the US forumers have an advantage, actually it always seemd to me that I was able to reply to a lot of their questions *before* the other US forumers started reading/answering... (but then, I usually hang out at the forums at night).

Jeff raises a valid point there: the one who asks is usually NOT the one to judge the correctness of the answer! Of course s/he can judge wether they can understand the replies and wether what s/he then did was helping him/her - but NOT wether the answers were technically correct or not!

And I agree with James that we should correct replies which are incorrect/dangerous (according to our knowledge). Maybe be simply warn and describe how to judge yourself, maybe be add the "right" answer. I have been corrected and I did correct somebody else's answer - and hey, nobody was complaining!

But I would like the option to *search* for threads with a minimum point-level... (maybe I should tell Dan in his monthly thread).

What the whole point-system is lacking is a way to add "confirmation" to replies: Yes, I agree, it worked for me
Imagine you have three different replies (and you do not understand the background for any of them), but one has ten "confirmations", and one has two "bewares" (did NOT work for me) - which would you investigate first???

Just my $0.0000002,
Wodisch