CFP last date
01 January 2025
Call for Paper
February Edition
CAE solicits high quality original research papers for the upcoming February edition of the journal. The last date of research paper submission is 01 January 2025

Submit your paper
Know more
The week's pick
Responsive image
Property (Land) Registration Management Using Blockchain in Nigeria

A. Festus Osuolale Ojebiyi David-Daniel Seriki Oluwasogo
Random Articles
Reseach Article

A New Criterion for Evaluating News Search Systems

by Mohammad Ubaidullah Bokhari, Mohd. Kashif Adhami
journal cover thumbnail

Communications on Applied Electronics
Foundation of Computer Science (FCS), NY, USA
Volume 2 - Number 7
Year of Publication: 2015
Authors: Mohammad Ubaidullah Bokhari, Mohd. Kashif Adhami
10.5120/cae2015651806

Mohammad Ubaidullah Bokhari, Mohd. Kashif Adhami . A New Criterion for Evaluating News Search Systems. Communications on Applied Electronics. 2, 7 ( August 2015), 28-35. DOI=10.5120/cae2015651806

@article{ 10.5120/cae2015651806,
author = { Mohammad Ubaidullah Bokhari, Mohd. Kashif Adhami },
title = { A New Criterion for Evaluating News Search Systems },
journal = { Communications on Applied Electronics },
issue_date = { August 2015 },
volume = { 2 },
number = { 7 },
month = { August },
year = { 2015 },
issn = { 2394-4714 },
pages = { 28-35 },
numpages = {9},
url = { https://www.caeaccess.org/archives/volume2/number7/411-2015651806/ },
doi = { 10.5120/cae2015651806 },
publisher = {Foundation of Computer Science (FCS), NY, USA},
address = {New York, USA}
}
%0 Journal Article
%1 2023-09-04T19:40:49.019247+05:30
%A Mohammad Ubaidullah Bokhari
%A Mohd. Kashif Adhami
%T A New Criterion for Evaluating News Search Systems
%J Communications on Applied Electronics
%@ 2394-4714
%V 2
%N 7
%P 28-35
%D 2015
%I Foundation of Computer Science (FCS), NY, USA
Abstract

Measuring the effectiveness of web search engines had been widely studied for the past fifteen years and different methods have been proposed by the researchers. These studies helps in identifying the most effective search engine and are useful for both users at the personal level and search engine vendors at the business level. So in this paper, first we extensively review traditional web search evaluation methods under four major categories and then discuss the urge for news search evaluation. We discuss possible criteria and quality measures foe evaluating web-based news search systems. And finally we evaluate four news search systems under a new criterion-information richness, i. e., extracting the useful contents from search result record (SRR).

References
  1. E. M. Voorhees. Variations in relevance judgments and measurement of retrieval effectiveness. In Proc. of the 21 st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Melbourne, Australia, pp. 315-323, 1998.
  2. C. M. Kelly and G. D. Moulin. The web cannibalizes media, Technical report, The forrester group, May, 2002.
  3. C. Yu and W. Meng. Web Search Technology. In The Internet Encyclopedia edited by Hossein Bidgoli, Wliey Publishers, pp. 738-753, 2003.
  4. K. L. Liu, W. Meng, J. Qiu, C. Yu, V. Raghavan, Z. Wu, Y. Lu, H. He and H. Zhao. AllInOneNews: Development and Evaluation of a Large-Scale News Metasearch Engine. In Proc. of SIGMOD’07, Beijing, China, 2007.
  5. W. Meng, C. Yu and K. L. Liu. Building Efficient and Effective Metasearch Engines. ACM Computing Surveys, 34(1), pp. 48-84, 2002.
  6. C. W. Cleverdon, J. Mills and E. M. Keen. Factors Affectinng the Performance of Indexing Systems, ASLIB, Cranfield Research Project, Vol. 2, pp. 37-59, Bedford, UK, 1966.
  7. H. Chu and M. Rosenthal. Search Engines for the World Wide Web: A Comparative Study and Evaluation Methodology. In Proc. of 59 th American Society for Information Science, Baltimore: MD, USA, pp. 127-135, 1996.
  8. W. Ding and G. Marchionini. A Comparative Study of Web Search Service Performance. In Proc. of 59 th Annual Meeting of the American Society for Information Science. Vol: 33, Baltimore:MD, USA, pp. 136-142, 1996.
  9. H. V. Leighton. Performance of Four World Wide Web Index Services: InfoSeek, Lycos, Webcrawler and WWW Worm, 1996.
  10. D. Hawking, N. Craswell and P. Harman. Results and Challenges in Web Search Evaluation. Comput. Netw. Vol:31, pp. 1321-1330, 1999.
  11. S. Lawrence and C. L. Giles. Searching the World Wide Web. Science, 280: 98-100, 1998.
  12. H. V. Leighton and J. Srivastava. First 20 Precision Among World Wide Web Search Services. J Am Soc Information Science, Vol: 50, pp. 870-881, 1999.
  13. M. Ljosland. Evaluation of Web Search Engines and Search for Better Ranking Algorithms. In Proc. of SIGIR’99 Workshop on Evaluation of Web Retrieval, 1999.
  14. J. Clarke and P. Willett. Estimating the Recall Performance of Web Search Engines. In Proc. of ASLIB’97, 49(7), pp. 184-189, 1997.
  15. D. B. Meghabghab and G. V. Meghabghab. Information Retrieval in Cyberspace. In Proc. of the American Society for Information Science(ASIS) Mid-Year Meeting, San Diego: CA, USA, pp. 224-237, 1996.
  16. C. T. Meadow. Text Information Retrieval Systems. Toronto, Canada: Academic Press: 1992.
  17. C. Gurrin and A. Smeaton. Improving the Evaluation of Web Search Systems. In Proc. of the 25 th European Conference on IR Research (ECIR ’03), Pisa, Italy, In: Sebastiani F. editor, Lecture Notes in Computer Science, Vol: 2633, pp. 25-40, Springer: New York, USA, 2003.
  18. M. Samalpasis, J. Tait and C. Bloor. Evaluation of Information Seeking Performance in Hypermedia Digital Libraries, Interact. Comput., Vol: 10(3), pp. 269-284, 1998.
  19. Y. Shang and L. Li. Precision Evaluation of Search Engines. World Wide Web, Vol:5(2), pp. 159-173, 2002.
  20. L. Li and Y. Shang. A new statistical method for performance evaluation of search engines. In Proc. of the 12 th IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2000), pp. 208-215, 2000.
  21. C. Silversien, M. Henzinger, M. Marais and H. Moricz. Analysis of a very large web search engine query log, ACM SIGIR Forum, Vol: 33(1), pp. 6-12, ACM Press: New York, NY, USA, 1999.
  22. A. Spink, S. Ozmutlu, H. C. Ozmutlu and B. J. Jansen. U.S. versus Europran web searching trends, SIGIR Forum, Vol: 36(2), pp. 32-38, 2002.
  23. L. T. Su, H. Chen and X. Dong. Evaluation of web-based search engines from the end-user’s perspective: A pilot study. In Proc. of the 61 st American Society for Information Science, Vol: 35, pp. 348-361, Pittsburgh: PA, USA, 1998.
  24. M. P. Courtois, M. W. Berry. Results ranking in web search engines. Online: 23(3), pp. 39-46, 1999.
  25. J. Gwizdka and M. Chignell. Towards information retrieval measures for evaluation of web search engines, 1999.
  26. D. Hawking, N. Craswell, P. Bailey and K. Griffiths. Measuring search engine quality. Information Retrieval, Vol:4, pp. 33-59, 2001.
  27. A. Chowdhury and I. Soboroff. Automatic evaluation of world wide web search services. In Proc. of the 25 th annual international ACM SIGIR conference on research and development in information retrieval, Tampere, Finland, ACM Press, pp. 421-422, 2002.
  28. A. Singhal and M. Kaszkiel. A case study in web search using TREC algorithms. In Proc. of the 10 th International World Wide Web Conference, Hong Kong, pp. 708-716, 2001.
  29. L. Vaughan. New measurements for search engine evaluation proposed and tested. Information Processing and Management, Vol: 34, pp. 557-579, 1998.
  30. L. T. Su. Value of search results as a whole search engines by undergraduate students. In Proc. of the 62 nd American Society for Information Science, Vol: 36, pp. 98-114, Washington DC, USA, 1999.
  31. L. T. Su. A comprehensive and systematic model of user evaluation of web search engines: II An evaluation by undergraduates. Journal of American Society Information Science Technology, Vol: 54(13), pp. 1193-1223, 2003.
  32. A. Spink. A user centered approach to evaluating human interaction with web search engines: an exploratory study, Information Processing and Management, Vol: 38(3), pp. 410-426, 2002.
  33. Y. Nasios, G. Korinthios and Y. Despotopoulus. Evaluation of search engines. Report undertaken by the National Technical University of Athens on behalf of the European Commission and Project PIPER, 1998.
  34. D. Nahl. Ethnography of novices’ first use of web search engines: affective control in cognitive processing. Internet Reference Services Quarterly, Vol: 3(2), pp. 51-72, 1998.
  35. L. Li and Y. Shang. A new statistical method for performance evaluation of search engines. In Proc. of the 12 th IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2000), pp. 208-215, 2000.
  36. F. C. Johnson, J. R. Griffiths and R. J. Hartley. Devise: A framework for the evaluation of internet search engines. Library and Information Commission Research Report 100, 2001.
  37. M. M. S. Beg. A subjective measure of web search quality. International Journal of Information Science, Elsevier, Vol: 169(3-4), pp. 365-381, 2005.
  38. I. Soboroff, C. Nicholas and P. Cahan. Ranking retrieval systems without relevance judgments. In Proc. of the 24 th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, New Orleans, LA, USA, pp. 66-73, 2001.
  39. S. Wu and F. Crestani. Methods for ranking information retrieval systems without relevance judgments. In Proc. of the ACM Symposium on Applied Computing, Melbourne, Florida, USA, pp. 811-816, 2003.
  40. J. A. Aslam and R. Savell. On the effectiveness of evaluating retrieval systems in the absence of relevance judgments. In Proc. of the 26 th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Toronto, Canada, pp. 361-362, 2003.
  41. J. A. Aslam, V. Pavlu and E. Yilmaz. A statistical method for system evaluation using incomplete judgments. In Proc. of the 29 th Annual International ACM SIGIR Conference on Research and Developmant in Information Retrieval, Seattle, WA, USA, pp. 541-548, 2006.
  42. W. Hersh and E. Kim. The impact of relevance judgments and data fusion on results of image retrieval test collections. In Proc. of the 2 nd MUSCLE/ImageCLEF Workshop on Image and Video Retrieval Evaluation, Alicante, Spain, pp. 29-38, 2006.
  43. E. Amitay, D. Carmel, R. Lempel and A. Soffer. A scaling IR-system evaluation using term relevance sets. In Proc. of the 27 th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Sheffield, UK, pp. 10-17, 2004.
  44. F. Can, R. Nuray and A. B. Sevdik. Automatic performance evaluation of web search engines. Information Processing and Management, Vol: 40(3), pp. 495-514, 2004.
  45. R. Nuray and F. Can. Automatic ranking of retrieval systems in imperfect environments. In Proc. of the 26 th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Toronto, Canada, pp. 379-380, 2003.
  46. S. M. Beitzel, E. C. Jensen, A. Chowdhury and D. Grossman. Using titles and category names from editor-driven taxonomies for automatic evaluation. In Proc. of the 12 th International Conference on Information and Knowledge Management, New Orleans, LA, USA, pp. 17-23, 2003.
  47. Open Directory Project. http://dmoz.org/.
  48. A. Mowshwitz and A. Kawaguchi. Assessing bias in search engines. Information Processing and Management, Vol: 35(2), pp. 141-156, 2002.
  49. R. Nuray and F. Can. Automatic ranking of information retrieval systems using data fusion. Information Processing and Management, Vol: 42(3), pp. 595-614, 2006.
  50. H. Sharma and B. J. Jansen. Automatic evaluation of search engine performance via implicit user feedback. In Proc. of the 28 th annual international ACM SIGIR conference on research and development in information retrieval, Salvador, Brazil, pp. 649-650, 2005.
  51. R. Ali and M. M. S. Beg. Automatic performance evaluation of web search systems using rough set based rank aggregation. In Proc. of the First International Conference on Intelligent Human Computer Interaction 2009 (IHCI 2009), Springer (India) Publisher: Allahabad, India, pp. 344-358, 2009.
  52. Y. Rasolofo, D. Hawking and J. Savoy. Result merging strategies for a current news metasearcher. Information Processing and Management, Vol: 39, pp. 581-609, 2002.
  53. K. L. Liu, W. Meng, J. Qiu, C. Yu, V. Raghavan, Z. Wu, Y. Lu, H. He and H. Zhao. AllInOne News: Development and evaluation of a large-scale news metasearch engine. In Proc. of SIGMOD’07, Beijing, China, 2006.
Index Terms

Computer Science
Information Sciences

Keywords

News Search Engines Search Result Records Time-Sensitive Ranking.

Powered by PhDFocusTM