Survey Systematics

Another Word of the Day - psychometric

2009-09-17T08:47:00.000-07:00

"Psychometrics is a field in which practitioners attempt to measure psychological response like emotion, intelligence, or other mental-based abilities, of individuals. An individual’s test results are then compared to an average group’s performance in these areas. A key example is the typical IQ test, which measures intelligence quotient. The test score you receive on an IQ test is compared to the body of people who have taken the test previously, and a number is assigned (the score) as your IQ number. Theoretically, a higher number IQ translates to greater intelligence."

The above definition taken from Wisegeek.com

Word of the Day - rubric

2009-09-17T08:39:00.000-07:00

"A rubric is an authentic assessment tool used to measure students' work. It is a scoring guide that seeks to evaluate a student's performance based on the sum of a full range of criteria rather than a single numerical score. A rubric is a working guide for students and teachers, usually handed out before the assignment begins in order to get students to think about the criteria on which their work will be judged. Rubrics can be analytic or holistic, and they can be created for any content area including math, science, history, writing, foreign languages, drama, art, music, etc...

The rubric is one authentic assessment tool which is designed to simulate real life activity where students are engaged in solving real-life problems. It is a formative type of assessment because it becomes an ongoing part of the whole teaching and learning process. Students themselves are involved in the assessment process through both peer and self-assessment. As students become familiar with rubrics, they can assist in the rubric design process. This involvement empowers the students and as a result, their learning becomes more focused and self-directed. Authentic assessment, therefore, blurs the lines between teaching, learning, and assessment (Pickette and Dodge)."

This definition comes from the Kessesaw State University website. Check out the link to learn much more about rubrics.

Teacher Evaluation and Assessment

2009-04-29T08:40:00.000-07:00

I just read an interesting article about the teacher evaluation and assessment process. Being involved in the business of providing OMR teacher evaluation forms for educational institutions and associations, I sometimes wonder what it would be like to be on the receiving end of such an evaluation. Judging by some of the comments I have read on Twitter, (using #scantron as search criteria) it sounds like students use the process to punish teachers. This would be a powerful tool to 'out' really bad teachers, but the question of objective fairness comes to mind and I'm sure teachers and speakers feel some concern about the evaluation process. I'd be curious to hear your experiences.

In the afore mentioned article "Teacher Evaluation & Assessment", educational experts, Anthony Cody and David B. Cohen answer questions about teacher avaluation and assesment. You may find this helpful in administrating your evaluation process. Read it here. Please feel free to return here and leave comments about your concerns regarding the evaluation process.

ScanTRON

2009-04-07T06:48:00.000-07:00

I ran across this on YouTube and just had to post it here. Rick did a great job with this animation. I know you'll enjoy this,...if you're old enough to remember the Disney movie, TRON. It's nice to see someone make something interesting and fun out of something that usually seems quite boring. Hey, this is what we do! Design, print, scan and collect very important data from these forms. I think it's fun...don't you? ;-)

Leave your comments.

Birds of a Feather Flock Together - Cool Benefits of Twitter

2009-03-03T11:40:00.000-08:00

We can all find others with similar interests as ourselves. We just need to ask the right questions. The problem comes in finding where others gather and then to find the ones actually listening.

I have been hearing a lot of buzz lately about the benefits of Twitter, a new technology for sharing short(140 characters or less)comments(tweets). Twitter is growing exponentially. For example, in the last seven days there have been close to 2 million tweets posted to the Twitter cloud. These tweets express the momentary interests of the twitters tweeting. Some are silly. Some of them are expressions of frustration or requests for assistance. Many are short reviews of products and services including short URL links to blogs or company websites.

By using advanced twitter search criteria, you can filter these tweets to display only those that contain words related to your particular interest. This is exactly what a lot of companies are doing. They use specific search keywords to effectively listen for specific tweets that include their company name or products. Then they respond directly to these tweeters with an introduction to begin a new client relationship and offer assistance.

This very thing just happened to me today when I posted that I had sent out an email newsletter using Constant Contact. Within 3 minutes I had received a reply to my tweet from none other than Constant Contact informing me that they noticed I was using their product and offering me any assistance I may need via Twitter. How cool is that?

Since discovering this new paradigm for relationship building around a common interest, I have been creating twitter feeds based on what I do in the OMR and web survey business and adding these feeds to my Google reader so I can easily follow anyone who tweets based on keywords that apply to our business. When I see relevant posts, I can respond by offering our company's assistance in areas of form design, or printing, etc.

If you happen to be on Twitter and have any related interest in web and OMR form design, printing, scanning and reporting, please follow me. My Twitter username is bubblepeeps. I will follow you back. It would be beneficial for us to form a flock around the topic of OMR Scantron type forms and processes. I will be looking forward to seeing you on Twitter soon!

Survey Systems' Bubble Publishing Support Program and FAQ

2008-12-12T07:31:00.000-08:00

The Bubble Publishing Suite of software was bought up and taken off the market by Scantron years ago. The Bubble Publishing Suite (which consists of Form Shop, Scan Shop, and Report Shop)is no longer actively marketed. However, there are many of us out here who still love to use it. Even though Bubble Publishing has not been updated in many years, it still is respected as a very easy-to-use and intuitive program that can handle the many processes involved in OMR (Optical Mark Recognition) data collection projects such as form design, form scanning and reporting.

Survey Systems is known in the industry as the Bubble Publishing experts. As a result, we are now preparing a Bubble Publishing FAQ that will soon be posted on our web site for other users to visit and hopefully find some helpful tips to solve software and hardware related issues.

We would appreciate it if you as Bubble Publishing Suite users would send us your ideas and suggestions regarding issues and solutions that you have run across over the years. Please send your helpful suggestions to lynn@sur-sys.com. We may include your submission in our Bubble Publishing User FAQ.

As an alternative, you may just want to give us the priviledge of taking care of all the "under the hood" details of your next data collection project. Either way, thanks for giving us the opportunity of serving the you in the faithful Bubble Publishing Community!

The Survey Systems' Staff

Conference Evaluation Is a Critical Aspect of Conference Management

2008-11-10T08:28:00.000-08:00

An important part of conference management is the conference evaluation. Most association conferences are repeated annually so it is critical for the planners to make an assessment of the quality of symposium sessions, speakers and overall experience. It will be less likely that association members and guests will attend future conferences if their previous experiences are mediocre at best. Only by getting relevant feedback from those in attendance will you ever know how well your conference was received and who you can count on to provide quality presentations for your future conferences.

A good conference planner will have a clear understanding of their goals and objectives and have an effective review process in choosing speakers and session topics. These instructor selections often come from personal encounters but most come from the recommendations of others. Speakers, session topics, venues and amenities don't always provide the kind of quality desired and need to be weeded out. You can accomplish this by providing attendees with an easy conference evaluation tool that will assist you in making the necessary changes in environment, personnel and material that will assure a more successful conference in the future.

What kind of questions should you ask?
With a clear understanding of the goals and objectives of the conference in mind, prepare a comprehensive list of questions and associated measurable responses regarding individual speakers and sessions. Question responses will be either "Yes / No" or multiple level responses such "Excellent / Good / Fair / Poor" or "Strongly Agree / Agree / Neutral / Disagree / Strongly Disagree" to name a few. Search the internet for examples of conference evaluation questions. The measurable values usually would be higher the more positive they are as in Excellent being a value of 4 and Poor being a value of 1. The result would be calculated as a mean and provide you with a quick glance at the overall response to a given question. It is customary to provide an overall evaluation section in the survey to capture the general sense of the success of your conference which would include site location and amenities and the impact that the conference overall will have on an individual's career or practice. Some open-ended questions or comment sections should also be provided to give the attendee opportunity to more freely express their personal insights and observations. All of this data will be extremely helpful to you in planning future conferences.

What is the most widely used evaluation instrument?
At the present time, paper OMR evaluation forms are the most widely used conference evaluation instrument. They are often combined with a web version of the survey for those more inclined to use their computers in the evaluation process. It is helpful in this situation to provide wireless 'hot spots' at the conference site for immediate participation while things are fresh in their minds. However, many attendees will prefer to respond online when they return to their home or office. The data from both of these sources can be combined and the tabulated results put into a readable report generally containing such things as response counts and tabulated percentages and mean values for easy review. The OMR and web survey process is best facilitated by a company with the tools and experience. They can also assist you in preparing your questions, and designing, printing and scanning your evaluations and preparing your reports.

As one involved in conference management, you look forward to positive feedback from your attendees to assure you that things went according to your best laid plans. However, negative feedback is also very helpful in making sure you get the best resources for your next event and continue to grow into a healthy professional association. Make sure to set aside a part of your budget for the conference evaluation process and find a reputable company such as Survey Systems to help you. May your next conference be better than ever!

How To Do Data Entry with Scanners

2008-10-20T11:33:00.000-07:00

"Can I use scanners and scanning software technology for data entry"? Yes you can, but only if it contains the best of penmanship.

As one who deals with paper surveys with open-ended questions and comments, I know what it's like to try to capture hand written text responses from those whose penmanship is a bit cryptic. Wouldn't it be nice to be able to just run these through some kind of scanner and have a computer transcribe it? Dream on!

There have been some very powerful advances in scanning software that does a pretty good job of converting printed words into editable ASCII characters. Most of us have used OCR software that comes bundled with scanners. I recently discovered the power of Adobe Acrobat to convert scanned printed material to editable ASCII characters that even automatically picks the correct font of the original. If handwritten comments are legible with the best of penmanship, Acrobat may even convert some of that, too. The quality of the text capture results are directly related to the clarity of the text on the scan. Scanning a document in pdf format creates an image, so make sure that your resolution is adequate. I would recommend scanning with no less than 300 dpi. If the quality of the original is high, the converted pdf file will look very much like the original except that the characters will now be editable and even searchable. The technology isn't perfect by any means, but it will amaze you.

Sometimes, when working with hard to read comments, it works best to use specialized software like Scantron's eListen 6.0 survey software. As their website says, "eListen 6.0 eliminates tedious manual data entry by capturing and storing handwritten responses using Scantron's Clarity OMR and Imaging scanner". This software captures the pre-defined comment section as an image which you can then let the reader try to decipher for themselves.

Another option that has worked well for us in circumstances where there are many open-ended questions is to scan the entire paper survey form using a high-speed copier like our Ricoh. We scan single or double-sided sheets and save the file in a pdf format and either burn the files to CD or post them to our ftp site.

Of course, there are many businesses, like ours, that provide manual transcription services for data entry of hand written comments to be included in your reports. That's part of the fun that I get to do from time to time.

Is NCS data form the same as Scantron data form?

2008-10-15T13:30:00.000-07:00

Scantron Form is the most widely recognized term to describe Optical Mark Read (OMR) machine-readable forms. NCS data forms are basically the same thing. In fact, NCS was recently purchased by Scantron. Scantron and NCS is to OMR what Coke is to cola and Kleenex is to tissue. The Scantron Form is commonly used in the education establishment for testing, conference course evaluations, surveys and other data collection projects. But it need not be limited to education applications. Because OMR is still considered the most cost-effective and reliable method for getting data off of paper into meaningful reports, it remains the time-proven standard against which alternate data collection systems are compared.

NCS forms (also referred to as Scantron forms) have a unique appearance but the process of designing, printing and processing this form is still the same. It follows critical criteria for positioning, printing and scanning in order to capture the data accurately. All the steps in the process involve specialized software and hardware. Scantron provides both, but so do many other good companies. In fact, there are other sources of OMR products and services that are actually better and more affordable.

Scantron gets a good deal when their name is attached to a generic product like an OMR form. But the customer may or may not get such a good deal by the association. It's important to note that there are other options to investigate. The generic association doesn't make it the best. Biggest maybe. But not necessarily the best. Scantron/NCS produces some great scanners. We use them and sell them. But we also use a variety of other software not produced by NCS that works better in our estimation. So don't be easily influenced by the big names. Do your research. Ask around for advice from those who have had experience with many different products and services. Good things often come in smaller packages.

Online Evaluation Surveys

2008-09-26T09:08:00.000-07:00

There is a "green" trend in the survey business. People are slowly moving away from printed survey documents and towards online evaluation surveys. There are many advantages to this move. With the proliferation of WiFi in most locations where people congregate, wireless cards in laptops and WiFi enabled smartphones, more folks are opting for the convenience of technology in data collection. If you google 'online surveys' and you will find a plethera of online survey software and businesses to suit your every possible situation. It can be overwhelming to say the least.

I dare say that most data-centric researchers don't really want to have to learn the ins and outs of setting up and maintaining an online evaluation project. Considering all the details of buying and learning specialized survey software, designing survey forms in a browser environment, webhosting issues and costs, database construction, etc., can leave one's head spinning.

There are a few online survey specialists midst all the "do-it-yourself" packages that can take care of all the details for you. You simply supply the questions and they will do the rest. Don't get me wrong, the challenge of creating an online survey can be a rewarding learning experience and provide you with many new tools to achieve your data collection goals for future projects. I know, that's what I do. There are plenty of tools out there for you to choose from and free instructional information on the web. Most people, however, like to leave these things to the specialists. Trust me. You will definitely sleep better. We have a motto where I work where we view ourselves as your survey mechanics. "You drive!...We'll take care of the rest!"

OMR

2008-09-24T09:15:00.000-07:00

OMR is an acronym for Optical Mark Recognition. You can read more about OMR here.
OMR forms are used for collecting data. Preparation of this type of form involves several critical steps.

Obviously, one needs to thoughtfully determine the kind of data that needs to be gathered and layout the questions in a clear, understandable format. These questions are then sent to the designer to figure out the best way to present these questions in an attractive layout. The designer needs to understand how the OMR scanner reads this information and position the questions and related response bubbles in just the right position on the form to assure that the scanner can accurately read the information. When the form design or layout is approved by the customer, the electronic form is moved to the printing phase. It is critical that the printer also understands the importance of consistent positioning of the printed sheet according to the layout. Below is a grid layout that represents the critical positioning of an OMR form.
As you can see, there are several grid density options to use based on the amount of data required for the form.

The two basic elements of this grid are the timing marks and the response bubbles. The timing marks appear in one column on one edge of the sheet. These rectangular marks are viewed by the scanner as the sheet travels through it. A timing mark determines when to begin and end the viewing of the corresponding row of bubbles and record the results. A typical form can have up to 48 bubbles across the sheet. In order for the scanner to know what bubbles to look at, a file called a scanning definition needs to be created with the scanning software prior to the actual scanning process. This scanning definition tells the scanner which bubble or bubbles in the grid to look at if filled in, record the assigned value. These values or results are then added to an Excel type data file as each form is scanned. When the scanning is finished the results are then tabulated with the use of special reporting software.

If the quality of the original scan form has been compromised by sloppy layout or printing processes, then the scanned data and the tabulated results potentially may not be reliable.

OMR is a great time-saving process in collecting data but can also create a huge headache if not done correctly.

Software: Slugger: Issues: Printing Problems

2008-09-15T08:36:00.001-07:00

I just received a call from a customer in education who bought multiple copies of Slugger, a program that enables you to overprint personalized data on a preprinted OMR (Optical Mark Recognition) evaluation form. This customer has an evaluation form that is common to multiple campuses that needs personalization with data that is unique to each campus.

Slugger is an excellent program to accomplish this, but it does have a couple of bugs. The problem is that when an overprinting document is created (a .ofc file), it records the printer name and parameters in the first line of this file. If you open the .ofc file with another copy of Slugger on a different computer and try to print to a different printer, you won't be allowed to do so.

The workaround for this problem is as follows:

Create a new dummy .ofc file on the machine that you want to print from and add at least one grid.
Under the 'File>Print Setup' menu, choose your desired printer and set up the page size, orientation and tray settings and then save your document.
Open the original .ofc file created on the first machine in a text editor such as Wordpad.
Open up the the new file just created on the new machine in the same text editor.
Using copy and paste, replace the second line of the original file with the second line of the second containing the new printer definition information and save the document.

Now you will be able to print to the new printer and even change to other printers on that machine. Unfortunately, there will never be any more updates to this software to correct this glitch so this is the ONLY solution to this issue.

Even considering the weaknesses of this dated piece of software, we personally prefer to use, sell and recommend Slugger as our favorite software package to personalize OMR evaluation forms.

If you are interested in purchasing Slugger or if you have any other questions regarding it's use, please, give us a call at Survey Systems, Inc. 1-800-473-7188.

Survey Systems launches OMR blog

2008-09-12T10:05:00.001-07:00

September 12, 2008, Survey Systems, Inc. launched a new blog called SURVEY SYSTEMATICS.
This first blog entry comes more in the way of a press release to announce it's creation, drawing some attention from those in the OMR crowd. "What is OMR?" you ask.

OMR stands for Optical Mark Recognition. It's not like you have to know what it is to have ever used it. You probably have filled in those funny looking survey forms with the rectangular marks down one side of the sheet and the little bubbles that you color in to indicate your answers to given questions. I remember randomly filling in those bubbles on something called the Iowa Basic Skills Tests when I was young. ;-)

OMR is a technology used for data collection from paper based surveys and evaluations. It requires very precise designing and printing procedures and specialized scanning equipment for accurate data collection. This is not an inexpensive or simple process. You will need the knowledge, experience and equipment for the detailed tasks in preparing a course, instructor or satisfaction evaluation form or survey. Hopefully many will come to this site to learn about all the unique processes and issues related to OMR form design, printing, scanning and reporting. We will also review the various kinds of OMR software, scanners and services.

Check back often or subscribe to our RSS feed for future postings.