START Conference Manager    

ASIST 2012 Annual Meeting 
Baltimore, MD, October 26-30, 2012 


Search Strategy Effectiveness and Relevance of YouTube Videos
Donghua Tao, Prajakta Adsul, Ricardo Wray, Keri Jupka, Carolyn Semar and Kathryn Goggins

Monday, 6:30pm


Summary

Precision and recall are two widely accepted measures to evaluate topical or system relevance along with Information Retrieval (IR) system performance. Different search strategies can yield different precisions and recalls for the same IR system, search engine or website. Therefore, precision and recall can also be used to indirectly test search strategy effectiveness. YouTube® is a popular social media website which hosts the largest number of user-generated videos on the web. The YouTube® search engine only provides text words search function and search results are retrieved through the system matching these search terms to video descriptions, tags, comments, etc. Therefore, precision and recall can also be used to directly test YouTube® performance. This paper aims to test if a search strategy using multiple search terms is effective by analyzing the relevance of the retrieved YouTube® videos on Smokeless Tobacco Products (STP) and YouTube® provided video usage statistics and community engagement statistics. 

17 search terms were used to retrieve YouTube® videos on STP which were recorded in a master file. All duplicate videos from the retrieved results were excluded. Unique videos were then selected based on pre-defined inclusion and exclusion criteria set by the research team. A sample of 440 unique videos was randomly selected for data analysis. SPSS 19.0 was used for data analysis. Descriptive analysis, precisions and recalls, logistic regression, and odds ratio test results are reported. The study found that only a few search terms out of the total 17 terms are more effective in terms of relevance of the retrieved videos. Also, YouTube® provided video usage statistics and community engagement statistics did not have significant association with the relevance of the retrieved videos. Recommendations for potential solutions on designing more effective search strategies are provided.