Showing posts with label evaluation. Show all posts
Showing posts with label evaluation. Show all posts

Saturday, October 27, 2007

New SEO Evaluation: Johndaughters.com

This week, off hours, I have been working on a new SEO evaluation. John Daughters, a hypnotist and hypno-therapist. He has been working on getting a web presence for his business, and has an excellent website. The only problem is, he needs more hits on his site.

He came to me and asked if I would take a look at his website and give him some pointers. The site is written beautifully, by a master web designer. But there are some things that could be added in order to increase his search engine optimization, and increase his search ranking.

Back Links
The website is beautiful, but is missing one major thing: Back links. While the website is designed well, according to the tools at SEOChat.com he does not have anyone linking to his website. Now, back links are a major need for any website and his website is no different. What can he do?

One thing that can be used is Web 2.0 technology. Blogs, Wikis, and other web presence options can be used to increase back links. One thing that I think will work for John is an entry in WIkipedia.org. Of course, the entry would need to be topical, and benefit the community as a whole (with accuracy).

Keywords on Website
Flash on a website is wonderful, but only if the flash is backed up with text on the page. John's main website is all flash, and as such it doesn't have many keywords in the body of the site for recognition. Luckily, his other pages do. That being said, he has plenty of meta tag keywords. Why are keywords in the body so important? Because Google doesn't utilize the meta tags for ranking. So page content is necessary. What counts for page content? Image Alt tags! And they are friendly to the American Disabilities Act. ^_^

Videos, Videos, Videos!
Search engine optimization is more than just getting to the top of a search. It's really all about bringing more traffic to the website. Two great ways to bring more traffic to a website is to post videos to YouTube, and/or to post podcasts regularly. If you are a performer this is ideal. It's like a free audition for your potential clients! You also want to make sure they are made well, with great angles and topical scenes. These then bring recognition to your website, and hence to your business.

If you are interested in more SEO ideas and techniques, or want to understand how an SEO analysis can work, feel free to sign up for the SEO class at the University of Utah's Continuing Education website. I hope to see you in my class! ^_^

Thursday, June 07, 2007

The Quest for Standardized Certificates

The computer industry is truly a wonder. No other industry has so much innovation, grows so quickly, or comes up with such a diverse method of doing things. As such, the industry seems to lack specifics regarding how to best learn about the technology. Most people within the industry then learn by experimenting, after given some basic information from the developing company.

But what if the skills desired are not necessarily applicable to a single company, but range through an entire industry and has diverse applications? This becomes problematic, as the development of any training material or reference material can become outdated quickly, and yet the need for such material is overwhelming.

On such example is Search Engine Optimization. Currently, there are several programs out there that claim to offer certifications based on their own issuance, but there is no guarantee that the certificate is recognized as an industry standard. Because of this, Search Engine Optimization is treated like an art form, rather than a set of skills that can be applied in a given situation.

The Lack of Standardization

Standardization only comes from an overwhelming acceptance from the industry at large. This can either be done unilaterally (i.e., Microsoft Certification, Sun Certification, Red Hat Certification), through a cooperative that organizes themselves from the industry's base to form a set of skill standards (i.e., Linux Professional Institute), or from a set of recognized experts that determine the standard within their fields (i.e., standard college educational standards).

In the world of industrial training, the needs assessment is based on the skills required to do the job. Most companies conduct their own internal needs assessment, which will result in a job description. Some may even look at what other companies assign, and try to duplicate those requirements without truly understanding what the requirements mean. Either way, the needs assessment is completed based off of an individual company's needs.

How to Build Solidarity
Solidarity within an industry comes from an industry-wide recognition of the requirements for a specific position or skill. Once solidarity has been reached (i.e., network administrator tasks are identified), they can be customized based on the company's unique requirements and are more easily met by those looking to participate within that industry.

In order to gain that industry-wide recognition, it would be necessary to focus on building ties between the industry leaders, find the similarities, and focus on the skills that can bring about success as defined within the industry. This means conducting a massive task analysis across multiple companies in order to identify the standard skills that will benefit all.

The Daunting Task Simplified
While this may seem like an impossible dream, it can be achieved if identified in these generic terms:

1. What is the expected result?

2. What skills are necessary to achieve that result?

3. Which results and skills are unique to the company, and which are universal?

4. Which should be unique, and which should be universal?

5. How can the skills be best presented, so that those learning can apply them?

Once these questions are answered, a consensus has been reached to the point that training can be given, and that training has recognized industry approval. At that point, any certificate that comes with it can be seen as an industry standard. That is, provided that there is an evaluation system that can certify the skills can be practiced.

The Evaluation
All standard certificates come with some sort of standardized evaluation system. Whether it is the old reliable multiple choice test, or a more hands-on evaluation process, the results are the same. That person is now recognized by the industry to have the skills required to perform the job he was trained to do. Whether that be SEO Specialist, Linux System Administrator, or Underwater Basket Weaver, the evaluation system is necessary to identify what is being absorbed.

But that's not all! It's also an excellent way to evaluate how effective the training material is, or the reference material can be. It all comes down to the Bell Curve, that hated curve what was always incorrectly applied in High School. The Bell Curve is actually the shape of a graph showing the normal distribution where the mean, median and mode are all identical.

If the materials achieve a success rate that at least meets that normal distribution, then the material can be called a success. If it weights heavily below the acceptable level then there is something wrong with the materials or the evaluation method, and therefore needs review and revision. This is very much unlike the High School bell curve grading system, where the bell curve was applied after grading to assign grades. If that method were applied in professional training, it would do injustice to the skills required, as it does not accurately reflect learning.

Friday, December 15, 2006

Evaluation: How You Know The Work Was Worth While

Your adrenaline is moving out of your system now, the class is over. How did you do? Most trainers can "sense" a general feel in the presentation and participation from the learners, but what were they really thinking? Did they get it? Can they do their job better now than they could before they started the training?

If you are a trainer, chances are you were hired for a specific job: making sure learners work better/faster/smarter. As with any other job, chances are your boss will want a full accounting of your performance in this area. How can you prove that you have accomplished your goal in a way that's measureable, and easy to understand? You do this through evaluation.

There are a number of ways you can evaluate the success of your training, depending on how much time you have to prove your worth to the company. There are the direct, timely methods, and there are indirect methods as well. Let's take a look at them both, and see which is best for you.

Direct Evaluation Methods
These are commonly called "Tests", "Assessments", and "Surveys". Basically, you check to see how well the learner had performed at the beginning of the course, give them quick tests in the middle of the course to see if they understand each of the modules you are presenting, and then have a final exam that tests overall comprehension. This is probably the most traditional method of evaluation, and everyone is pretty much familar with it. But it only looks at a small snap-shot of the learner's abilities. You don't know if the targeted skills are going to be applied.

A real bonus from this method, particularly from the survey, is that you can get a feel for your development and implementation of the course. How did it appeal to your learners? How are you doing as a presenter? There are a number of things that you can learn that will add to your ADDIE development through this method, outside of just whether or not the analysis was correct.

Indirect Evaluation Methods
Indirect evaluation methods would include monitoring employee performance over a long period of time, focus on overall numbers and how they relate to the skill that needed to be taught. Is there an improvement? Did it warrant the devotion of resources?

For those who are familar with any type of research, this should be nothing new. Researching the results of a change is part of what analysts do, and makes them so valuable to companies (mostly because it's so boring no one else wants to do it ^_^). But what do you analyze? Focus on the results as compared to your initial needs analysis. Did the numbers you focused on for your initial analysis change? Did they change for the better? Where there other factors involved that were not initially recognized?

For those trainers that are caught in the political arena within your company and were forced to create a training program to compensate for non-skill related issues, this is a perfect time to emphasize that while the skill became better known, the outcome did not improve because of the x and y factors. If you provide the information in a scientific way, showing that even though the training was a success the solution failed to be realized, the management will often concede, or let you go, which would also be an acceptable alternative. Who wants to be blamed for someone else's incompetence?

Seriously though, it's a good method to see how effective your training was, your analysis was, and how well each of the learners assimilated the information. You learn how well things are going, how you can improve your teaching style, and therefore increase your effectiveness as an instructor. A success here will validate your work, give you a great promotion, raise, and a chance to win a free 2 week vacation in the Bahamas! ^_^

When to Use Your Evaluation Style
Neither evaluation method is perfect on it's own, so combining both is essential for a full view into how well you are doing. Use a quick assessment at the beginning of the course to find out where your learners are (if that is in question). Once you know, have them keep their scores for future comparisons and self-evaluation. Also have an after-class evaluation that is done anonymously away from the classroom environment. This way the instructor doesn't have a presence to influence the outcome of the evaluation.

Then, send two more evaluations, once after 3 weeks, and one after 2 months. This way you can find out how well the content is remembered, and what the percentage of recall is for the learners. This is good long term data to be gathering. And finally, spend some time doing indirect evaluations by checking performance numbers. Of course, this assumes that you have access to the information. If you don't, you may want to provide a quick spreadsheet to the company that contracted your services so that they can provide the final data to you. They can leave out any information that may be proprietary and still provide enough information to let you know if you have been successful in your endeavors.

So, that finishes this segment of the ADDIE program. I may post some additional information on the subject, but for now, I wish all of you good luck in your training development!

Monday, December 04, 2006

ADDIE in Corporate World Training

In the corporate world, Training can be viewed with a lot of doubt, particularly when those who are less experienced with training and more with the subject matter are delivering it. As a manager of mine had once said, "Subject Matter Expert's are not Trainers". Why would that be so? SME's are often more knowledgeable than a trainer is, and therefore can provide greater insight into an issue than a trainer can. So what makes a trainer so special, and in some cases higher paid, than the SME? One tool a Trainer has over an SME is the ADDIE system.

The ADDIE System
The ADDIE system of curriculum development is what separates a SME from a trainer. That is, the Analysis, Development, Design, Implementation, and Evaluation of the content (ADDIE) can be applied to any subject, any situation, and build deliverables that are directly targeted to the skills that are required. By using this process the trainer can build a framework for the instructor (not always the trainer) to follow while offering their training course. So let's look at this development concept more closely.

Analysis
This is by far the most time consuming process in training development. Analysis is not just an analysis of what needs to be taught, but whether or not training as a whole is the answer. Just because there is a deficiency in one area doesn't mean that better or more training will fill the void.

Training only helps people that do not know what they don't know (unconsciously incompetent). Training allows people to know what they don't know (consciously incompetent), and help them identify the tools that will add the skills to what they know (consciously competent).

Other potential problems that Training cannot resolve are management issues, unrealistic expectations, undefined requirements, poorly developed tools, etc. These are issues that need to be addressed in other areas, and training can't influence. Poor management (i.e., "The Boss" from Dilbert) can completely undercut real performance and development from the consciously competent level to the unconsciously competent level (where we all strive to be). Unrealistic expectations and undefined requirements leave the employees in a sort of limbo, which impacts performance heavily. Also, if the tools are inferior, there is only so much the employee's abilities can do to overcome it.

Training is specifically geared to understanding and comprehension. If the problem you have is related to this area only, then a training course can be the answer to your needs. But what needs to be trained?

Analyzing the job is definitely necessary. Every Duty, task, sub task, and skill needs to be identified. By identifying the work process in this manner, training can be developed from the ground up. But be warned, this is a time-consuming process best suited to those personalities that can take that much analysis. As such it can tack on hours of preparation to the actual training process. The good news is that any properly run Human Resources department should have done this for you. If not, get ready to break down the task as best you can. And once it's done, be sure to archive it so you never have to do it all again (just update it as it changes).

Development
Once you have waded through the Analysis portion, it's now time to start developing the course. Development means creating an outline, getting the learning objectives stated, and identifying the assessment points that assess the learner's acquired knowledge.

The outline is basically the final analysis document, but organized in the learning pattern everything will be presented. Generally, the course topics will build upon each other, as well as allow the learner to build upon what they already know. This is known as the Constructivist theory of learning, and is the most common method of teaching being used today.

The learning objectives outline what is expected to be learned after the section is complete. This is directly related to the assessment points, as each point needs to be satisfied in order to complete the learning objective.

Design
Training is, ultimately, a form of entertainment. It doesn't matter how well someone knows their stuff, if they can't keep the attention of the learner then it was all for naught. The design portion is a way to outline the course, much like blocking out a scene on the stage. Design includes how the content is deployed, and therefore which learning style is focused on. It can be a hands on trial (for tactile learners), written content (for auditory learners), or visual presentations that catch the eye in an appealing way (for visual learners). A good trainer will utilize at least two if not all of these design segments in order to appeal to all the potential learners out there.

Here I spend time creating participant guides, instructor guides, gathering multimedia presentations that apply to the course, etc. There is a lot that can be done in the Design phase, and luckily it's generally the most fun.

Implementation
Finally, after all that work, the instructor can finally implement the course. If the course was designed correctly, anyone with the appropriate skills and knowledge can take the materials and teach the class, as long as they have some basic delivery skills. As I had said before, training (or teaching for that matter) is all about the performance. Read your learner audience. Are they responding? Do you get blank stares? Do you perhaps need to crack a joke to see if they are breathing? There are a lot of techniques that each trainer likes to use, depending on the subject or audience. Find what works for you, but be willing to be flexible in your delivery.

It's also important to let the learner know when you don't know an answer. Don't blow it off, make up an answer, or through the question back at the learner for asking something you don't know. What you do is tell them you will find out. I can't tell you how many times a learner has been given bad information by a so-called "trainer", and have it adversely impact their job. And once the bad information or skill is learned, it takes a lot more work to correct the learner and get them on the right path.

Evaluation
Evaluation should, honestly, be conducted throughout the course creation process. At every stage the content can be evaluated for quality and effectiveness. But, ultimately, the best evaluations are conducted after the training is over. Here is where you get to find out how well the course was taught, and whether or not the training accomplished what it was set out to do: fix the deficiency.

That, in a nutshell, is the ADDIE program. There can be a lot of detail added into each section, but overall this is how it works. Course development can take months of planning, analysis, development, etc. All so that a skill is learned, which increases the success of the learner, and therefore increases the success of the employer of the learner.

Does this mean SME's can't do it? Of course not! SME's can learn to use the ADDIE process, but once they do they become Trainers. At least to me, they do. And if a "trainer" doesn't use this process, they are just a glorified SME, and perhaps not even a SME at that.