What Our Students Said – Usability Testing Recap
In mid-April, University Communications & Marketing (UCM) and Information Technology (IT) conducted usability testing with current USF students. The testing took place over the course of two days in the Library with great results.
So, what was the purpose of the testing? To make sure the site works as intended of course! Along with the new CMS, colleges, departments, and programs will have access to many new templates to build their sites. We needed to see how students interacted with the site and what they were looking for.
What was the process?
UCM designed several templates, which were used to develop a “test site” that would allow the students to click and students to click and navigate through just as they would on a live site.
Concurrent with the template development, the following items were created to conduct the testing:
- Task List: 10 tasks based on common scenarios performed on the current site were listed and printed out to give to the testing participants.
- Script: information that was read to the participating student that fully explained the process and what was expected of them. It also outlined steps for the facilitator.
- Video Release Form: to be signed if the student agreed to be videotaped while going through the test.
- Observation Form: a basic form that the facilitators used to take notes during the test itself. They were able to mark if the participant completed the task, if students had any issues, or if there were any comments that arose while trying to complete the task.
- Exit Survey: 14 questions that the students were asked to answer at the completion of their experience.
What was asked?
The first thing the students were asked to do was to spend a few minutes looking at the homepage and say what they thought about it; what they liked or didn’t like, what struck them, etc. Once that was completed, they were given the below task list and asked to talk their way through as they tried to do each item.
- Find information regarding Art History.
- Find an Art History class.
- Locate where you would find the cost to attend.
- Find Student Government information.
- Locate where you would schedule a campus tour.
- Find contact information for an Architecture adviser.
- Locate student employment information.
- Locate the Tutoring Center.
- Find Spring 2012 final exam dates.
- Locate where you would find admission requirements for graduate school.
At the conclusion, they had the opportunity to ask the facilitator any questions they had and then completed the exit survey. The survey asked questions regarding navigation, design, and organization along with what information students would like to see on the homepage, areas that could be improved upon, and any additional comments they had.
What was the feedback?
Very favorable! Below are just a few comments that were heard regarding the homepage:
- Good branding
- Brighter than old site
- Good organization
- More aesthetically pleasing than current
- Cleaner and more professional
- Engaging
- Clear navigation
- Very clean and visual
- More modern
- Visually striking
A brief video of some the student comments have been created for viewing. A longer, more detailed version is being produced and will be posted soon.
What are the next steps?
A lot of the feedback that the students shared has already been used to refine the design templates. Compilations documents and videos from the testing will be shared with any appropriate groups throughout campus. More usability testing will be conducted among relevant audiences in the coming months and even after the new CMS is officially launched.
One Step Closer
April 19, 2012, 8:22 am
Filed under:
CMS
The cut-off date outlined in the Invitation To Negotiate (ITN) for vendors to submit their bids has passed and the reviewing stage has begun. The Selection Committee will be narrowing down the prospects to the top three with an award still scheduled to take place May 7. As previously stated, once a CMS vendor has been chosen, details will be worked out on implementation timelines and procedures. At that point, a tentative production schedule that details the order in which university colleges and divisions will move into the new content management system will be created.
University Communications and Marketing has continued to refine and present drafts of the template design to multiple groups and leaders throughout campus. A test site was completed and usability testing has begun! UCM and IT surveyed student volunteers Monday, 4. 116 and Wednesday, 4.18 in the Library and received great feedback from students who participated in the testing. Plans are also underway to schedule time with high school students during the summer in order to get real, “prospective” student opinions on the site.
At the conclusion of the first phase of usability testing, UCM and IT will review all of the feedback and make any appropriate changes to the templates. Additional usability testing will take place again as the CMS gets closer to launch.
In other news: Technology Fee season is here!
It’s that time of year again! March 6th was the deadline to submit a proposal for the 2012-2013 Technology Fee cycle for System and Tampa campus, and IT received 30 proposals with a total funding request of over $12 million.
Some projects that are being considered for funding include a mobile Blackboard application for students, more wireless upgrades around campus, and more! These proposals will be reviewed over the next few weeks by the different advisory committees, including the Student Technology Advisory Council.
If you have any questions about the Technology Fee, please contact technologyfee@usf.edu or check the Information Technology site for updates.
ITN, Usability Testing, and More!
March 7, 2012, 10:18 am
Filed under:
CMS
University Communications & Marketing has been working on creating the first drafts of the template design that all sites will follow for non-academic entities, academic entities, and the main USF site. Meetings have been conducted with a few small departments to present the preliminary template design resulting in positive feedback. UCM will continue designing templates which will then be formally presented to larger groups once more details of the CMS vendor have been determined.
Although the CMS demos have been completed, the formal process of selecting a vendor has just begun. UCM, IT, and Purchasing have been working together to finalize the Invitation to Negotiate (ITN) document to post for vendors. The ITN is a mandatory process that must be done in order to procure a CMS vendor. The ITN document provides information to the vendors on requirements and asks them to provide the best solution for our needs. This process also allows the Selection Committee to review proposals that all have consistent formatting making it more efficient to evaluate.
The initial timeline for the ITN process is as follows:
- March 6: ITN release
- March 16: Vendor questions due
- March 23: USF responses due
- April 2: Proposal deadline
- Mid April: Evaluations complete and Presentations
- May 7: Bid awarded
Once a CMS vendor has been chosen, details will be worked out on implementation timelines and procedures. At that point, a tentative production schedule that details the order in which university colleges and divisions will move into the new content management system will be created.
Concurrently with the ITN process, UCM and IT will be building a test site to allow for usability testing. The testing and evaluation will give the team direct input from our target audience on how real users will use and navigate through the site. Testing will consist of creating five to ten scenarios for the participants to follow. The user’s process will be observed and they will be asked to complete an exit survey on how they feel the site performs. This is tentatively scheduled for mid April to allow students to participate before final exams.
FAQ’s and Next Steps
February 2, 2012, 12:59 pm
Filed under:
CMS
Our challenge:
The current USF website is made up of hundreds of disparate sites that seem to have little relation to one another. Many of these sites are filled with out-of-date information, broken links and inappropriate content. Unfortunately, even relevant content is often so poorly organized that people who visit the website find it difficult to get the information they need.
So how do we fix it?
The first step is recognizing that the USF website is a marketing tool—in fact, it’s USF’s number one marketing tool. We all need to remember that the purpose of the USF website is market the university to our primary audience: prospective students.
We know that prospective students are looking for very specific information when they visit university websites. Primarily, they are looking for information about academic programs, admissions requirements, and cost to attend / financial aid.
Prospective students are also interested in knowing what kinds of jobs they will be qualified for after they complete a specific degree program, as well as information about campus life, housing, dining, parking, internship opportunities, student employment and other resources.
Finally, prospective students enjoy viewing videos and photos of the campus, learning about student activities and events, and reading news stories and profiles about faculty, current students and alumni who have found success at USF.
Are there other audiences besides prospective students?
Of course. We also need to market the university to parents, donors, alumni, job seekers, legislators, vendors, patients, event attendees and other individuals who interact with and support the university.
Note that all of these secondary audiences are EXTERNAL, as opposed to our internal audiences, which include current students and university employees. While the USF website will certainly include content relevant to current students and employees, such as maps and dining venue hours, INTERNAL information for these audiences should not be posted on the USF website.
If I can’t put internal information for current students and university employees on the USF website, then where CAN I post it?
With few exceptions, nearly all content for current students and university employees belongs in the MyUSF portal. Within the portal, students can communicate with faculty, access / attend online classes, and conduct a wide variety of university business, such as checking the status of their financial aid awards and monitoring their progress to graduation.
Likewise, employees should conduct all university business within the portal, including accessing enterprise business systems such as GEMS and FAST, searching for Human Resources forms, monitoring leave balances and much more.
Are there other types of content that DO NOT belong on the USF website?
Yes. Instructional content—course-related content produced by faculty for current students—does not belong on the USF website. Information Technology will provide space on separate servers for this type of content.
In addition, applications—also known as online forms and the databases that support them—do not belong on the USF website. Again, Information Technology will provide an appropriate solution for this content, to be determined on a case-by-case basis.

(click to enlarge)
I understand that I need to edit my content. How do I begin?
The first step is to carefully review your current web content. There are two goals here:
- To eliminate all out-of-date, inaccurate content.
- To relocate any content that is not appropriate for the website, such as content for internal audiences, instructional content and applications. (See above.)
The next step is to think about what you are trying to achieve. Do you want to motivate your audience to apply for a job, purchase a ticket, or pay a bill? Then you need to develop content that motivates your audience to take those specific actions.
This means that every word, link, photo, video and graphic element on every web page should serve a very specific purpose—to motivate your target audience to do what you want them to do.
It also means that any extraneous information needs to be edited out.
But isn’t it helpful to include tons and tons of additional information?
No. One of the most significant problems with the current USF website is that it is filled with irrelevant content. When you feature multiple navigations, long lists of “related links” and broad paragraphs of text, you make it difficult for your audience to find the information they need to take action.
Redundancy is another significant problem on the current USF website. When you post the same set of links in four different places on one web page, or post the same content over and over again on multiple pages, you confuse your audience.
Likewise, when you recreate content that already lives somewhere else on the USF website—say, parking information—you not only clutter up your web page, but you also run the risk of failing to note when the original content changes, leaving you with a web page filled with out-of-date, inaccurate information.
What do I do after I edit my content down to the essentials?
Now it’s time to get organized. Stay tuned to our next blog post for more information.
What’s next?
Information Technology and University Communications and Marketing are currently creating a production schedule that details the order in which university colleges and divisions will move into the new content management system.
Several units have already volunteered to go first, among them the College of Education and Human Resources.
If you would like to add your unit to the queue, or if you have additional questions about this initiative, contact Kathleen Long at kathleenlong@usf.edu, or Stephanie Harff at sharff@usf.edu
UCM and IT have been presenting the Web Re-branding/CMS Communications to departments and colleges across campus:
Presented to:
- Provost’s Retreat
- Council of Deans
- ACE Council
- A&P Council
- CTIR
- ITMC
- STAC
- Professional Communicators Group
- IT Standards Board
Met with:
Academic
- College of Arts and Sciences
- College of Marine Science
- College of Education
- College of the Arts
- Patel Center for Global Sustainability
- Air Force ROTC
- Undergraduate Studies
- Office of Student Success
- Physical Education Department
Non-academic:
- Administrative Services
- HR
- Student Affairs
- USF System
- IT
- UCM
- Media Innovation Team
- Provost’s Office
- SDS
Areas using the Interim Solution:
- Wellness Education
- Campus Recreation
- Undergraduate Admissions
- Computer Store
- Ethics Committee
- ATLE
- University College and College of Ed
- Housing
- Business and Finance
- iWINLAB – Engineering
- Budget and Policy Analysis
- Department of Child and Family Studies
- Admissions
Advisory Committee
- College of Business
- College of Education
- College of Engineering
- Library
- Office of Research and Innovation
- College of Arts and Sciences
- Office of Undergraduate Admissions
- College of the Arts
- Administrative Services/HR
- Provost’s Office
Demos!
January 23, 2012, 2:16 pm
Filed under:
CMS
Vendor demos are underway! As you may recall, the CMS Workgroup whittled down the list of almost 30 CMS solutions to nine, and seven demos have been scheduled:
- doCMS – 1.11.12
- ektron – 1.18.12
- SiteFinity – 1.25.12
- Alfresco – 2.1.12
- OmniUpdate – 2.7.12
- Jadu – 2.8.12
- TerminalFour – 2.8.12
From this list, the top three will be selected, and then the final CMS solution will be picked.
Right Around the Corner
December 20, 2011, 10:22 am
Filed under:
CMS
So what do we have planned next year?
Vendor demos will take place throughout the month of January. Demos will include an overall look at the CMS, as well as a Q&A that will allow attendees to ask specific questions, with the goal of ensuring that the final solution meets the needs of the university community.
The nine finalists include:
If you have any questions, or would like to be invited to attend the vendor demos in January, please email Carolyn Mourey.
A Few Updates
December 15, 2011, 10:36 am
Filed under:
CMS
During the past few weeks, the CMS Workgroup evaluated 28 potential CMS solutions, and has narrowed the field to nine products. The solutions were evaluated using a detailed matrix, and the nine finalists scored high in terms of meeting project requirements.
The Workgroup also developed preliminary wireframes, which they plan to review with the advisory committee in January. The wireframes will serve as the basis for template development, which will begin in early 2012.
The CM Advisory Committee has new members. The new members include Kim Tucker, CMO, College of Education; Kelley Curtis, Webmaster, College of Education; Hal Dunn, Web Designer, Research and Innovation; Jennifer Carlevatti, Associate Director, Undergraduate Admissions. We appreciate their involvement and look forward to their input.
Weight and Rank
December 1, 2011, 8:26 am
Filed under:
CMS
Requirements Finalization: Done
Features Checklist: Done
List of Solutions: Done
Wireframes: Almost Done!
Major headway has been made the last few weeks regarding the Great CMS Project. A lot of the goals set out for the first portion of the project have been met and now the group is moving towards finding a solution. Using the finalized requirements document and features list, each group member evaluates three CMS solutions. Members compare the requirements to what each CMS offers and ranks them accordingly: 3=High; 2=Medium; 1=Low; 0=N/A.
The weighted score is then compared to all the products, and the top 3 will be brought in for a demo.
There are many products out there, and by having the CMS Workgroup rank the requirements they tirelessly brought together, USF will be able to find a great CMS solution to fit the needs of the community.
Coming Soon: Solutions!
November 7, 2011, 9:11 am
Filed under:
CMS
It has been a busy few weeks for the CMS Workgroup!
Our members have been reviewing the feedback provided by the CMS Advisory Group from the Oct. 7 meeting, and are adding relevant recommended items to the requirements document. Once the workgroup has finished, the document will be finalized and reviewed by the Vice President of Information Technology, Mike Pearce.
In preparation of the final document, the Workgroup is creating a very concise checklist to use when comparing different CMS programs/solutions that are currently available. The goal of the checklist is to provide an easy way for group members to see what certain programs/solutions have and compare it to what we need (as noted in the Requirement document).
Everyone is very excited to be approaching these next few steps in the Great CMS Project!