I let the students in my genetics-pilot class choose a topic for me to teach about, and they chose The Genetics of Behaviour. The class comes up next Monday (the second-last class of the term), so I need to get to work on it. Of course I know nothing about this, but that's not a big problem - I can certainly learn.
The problem is that all I've found is various individual studies that connect particular behaviours with particular genotypes. All very well, each reasonably good science about an interesting behaviour, but essentially just a series of anecdotes. What I'm hoping to come up with is a lesson, or a take-away message, or some unifying principle.
What form might such a principle take? That behaviour is a biochemical phenomenon? That it isn't? That we still have free will? That we don't? That nature and nurture interact to determine behaviour, as everything else?
Maybe I should start the class by raising the big questions (after I start preparing for the class by finding out what these are). What is consciousness? No, too big. Do we have 'free will'? Also too big. In fact, I think both these questions should be better handled by saying "What do we want the words 'consciousness' or 'free will' to mean?" After we've settled on clear definitions we'll be in a better position to answer questions about them.
So I won't start with such big questions. How about 'What can genetics teach us about why we behave the way we do?" A bit vague. Should I pick one behaviour that's been well studied and report on the findings? Not if it's a boring behaviour like the ability of rats to remember the location of an underwater platform. What about all those Drosophila vegetable mutants (rutabaga, cabbage, turnip, dunce)?
Sexual behaviour is much more interesting, especially to college kids. And it's where natural selection/sexual selection has probably acted very strongly, but in complex ways. Maybe the genetics of sexual orientation - review the studies, tell them whether there is any good data at all? Wikipedia looks like a good starting point, but it tells me that there's very little good data, certainly not enough to base a whole lecture on.
What about human pheromones? The whole question of whether we pick our mates partly because they have different HLA alleles than we do. All those T-shirt-sniffing experiments are thought to be detecting genetic differences. And there's that ocytocin study, and lots of sleazy marketing. And then I could end with the new Drosophila study showing that the smells being used for mate choice can come from bacteria which come from the food source...
Monday, March 28, 2011
Tuesday, March 15, 2011
The 'calibration' part works!
My students have finished their Calibrated Peer Review assignment, and now I'm discovering the wealth of instructor resources for interpreting the outcomes and correcting discrepancies.
First is a Student Results page, listing each student's total score on the assignment (text plus reviewing activities), the points earned by the text they submitted, and their Reviewer Competency Index. The latter is a measure of how well they reviewed the three calibration texts, and is used to modulate the ratings they assigned to the three student submissions they reviewed. From this screen you can click on any student's name to see all the details of the scores they assigned and received.
Next is a Problems List page. This again lists the students, this time flagging in red any problems the system noted with their assignment. Students who failed to complete the calibrations on time are flagged and their Reviewer Competency Index is zero. Reports that were reviewed by only a single student, or not reviewed at all, are flagged. So are reports that received discordant reviews (discrepancies exceeding a pre-set tolerance) , and reports that were reviewed by students with very low Reviewer Competency Indexes.
Only two reports had seriously discordant reviews, and rather than being evidence of problems with CPR, both of these demonstrate how well the CPR system works. The first was a very good report that received one very low score because the reviewer had given too much weight to very minor flaws. But because this reviewer had also performed badly on the calibration reviews, they had a low Reviewer Competency Index and this poor review didn't drag down the good report's score. The second problem report also had one very low score and two good scores. But this time the low score was from a very competent reviewer, and when I read the report I discovered strong evidence of plagiarism, which would certainly justify that low score. (The other two reviewers evidently assumed that the professional writing was the student's own work.)
The third set of resources are on a Tools page. Here you can change deadlines for individual students to allow them to submit after the original deadline has passed (I've done this for one student), change student's ratings and scores, and have the whole assignment remarked to incorporate adjustments you've made to a Reviewer Competency Index. You can also download the complete texts of submissions and evaluations. You can even access the system as if you were any one of the students (this gets you a caution that any changes you make will appear to have been made by the student).
Overall I'm very pleased with the CPR system. I've only encountered a few minor problems: One is that many of my students earned low Reviewer Competency Indexes. I suspect this is because I had so many calibration questions about specific details of the submissions. Several students had minor problems submitting the various components by their deadlines. I think these were mostly because they overlooked a final step needed to complete the submission. The text-only interface didn't seem to cause many problems - some students had trouble with the word limit (because HTML tags are counted as words), but only one student's submission had text-formatting errors.
Now it's time to start spreading the word around campus about this new resource. Hopefully there'll be enough interest that UBC will decide to purchase it once our free trial ends.
First is a Student Results page, listing each student's total score on the assignment (text plus reviewing activities), the points earned by the text they submitted, and their Reviewer Competency Index. The latter is a measure of how well they reviewed the three calibration texts, and is used to modulate the ratings they assigned to the three student submissions they reviewed. From this screen you can click on any student's name to see all the details of the scores they assigned and received.
Next is a Problems List page. This again lists the students, this time flagging in red any problems the system noted with their assignment. Students who failed to complete the calibrations on time are flagged and their Reviewer Competency Index is zero. Reports that were reviewed by only a single student, or not reviewed at all, are flagged. So are reports that received discordant reviews (discrepancies exceeding a pre-set tolerance) , and reports that were reviewed by students with very low Reviewer Competency Indexes.
Only two reports had seriously discordant reviews, and rather than being evidence of problems with CPR, both of these demonstrate how well the CPR system works. The first was a very good report that received one very low score because the reviewer had given too much weight to very minor flaws. But because this reviewer had also performed badly on the calibration reviews, they had a low Reviewer Competency Index and this poor review didn't drag down the good report's score. The second problem report also had one very low score and two good scores. But this time the low score was from a very competent reviewer, and when I read the report I discovered strong evidence of plagiarism, which would certainly justify that low score. (The other two reviewers evidently assumed that the professional writing was the student's own work.)
The third set of resources are on a Tools page. Here you can change deadlines for individual students to allow them to submit after the original deadline has passed (I've done this for one student), change student's ratings and scores, and have the whole assignment remarked to incorporate adjustments you've made to a Reviewer Competency Index. You can also download the complete texts of submissions and evaluations. You can even access the system as if you were any one of the students (this gets you a caution that any changes you make will appear to have been made by the student).
Overall I'm very pleased with the CPR system. I've only encountered a few minor problems: One is that many of my students earned low Reviewer Competency Indexes. I suspect this is because I had so many calibration questions about specific details of the submissions. Several students had minor problems submitting the various components by their deadlines. I think these were mostly because they overlooked a final step needed to complete the submission. The text-only interface didn't seem to cause many problems - some students had trouble with the word limit (because HTML tags are counted as words), but only one student's submission had text-formatting errors.
Now it's time to start spreading the word around campus about this new resource. Hopefully there'll be enough interest that UBC will decide to purchase it once our free trial ends.
Saturday, March 12, 2011
Peerwise let my students write their own midterm problems!
(Of course they didn't know it at the time!)
The students in my introductory genetics pilot class have had weekly Peerwise assignments all term. In alternate weeks they either had to either create at least one multiple-choice genetics problem suitable for an open-book exam (i.e. not based on memorization) or to answer and critique at least two problems previously created by other students.
I had used Peerwise a couple of years ago in a first-year class. Each student only had to create one question and critique four. I had found their questions to be full of confusions, poorly written and lacking important information, but the critiques were quite good.
This time, my students have told me that they don't mind answering the problems other students have posed but find creating their own to be quite difficult. The first problems I had looked at hadn't been very good. But last week I started going through their more recent problems, looking for ideas I might use in creating problems for the upcoming midterm, and I was pleasantly surprised by the high quality of many of the problems they had developed.
I was so impressed by the students' questions that I decided to use them for the midterm, rather than writing my own. I downloaded a range of questions to consider - some of them were very good but much too difficult for a 45-minute midterm. I had thought that the good questions might have all been written by one or a few students, but when I checked their usernames I found that 16 of the 17 questions had been written by different students; this means that most and maybe all students are creating good problems!
I used 14 questions for the exam - most of them I was able to leave unchanged, but some required minor editing for clarity. The students must have recognized their own questions, but I haven't seen anything on the course discussion board to suggest that they've realized that they collectively created all the questions on the exam. I'll tell them on Monday.
The students in my introductory genetics pilot class have had weekly Peerwise assignments all term. In alternate weeks they either had to either create at least one multiple-choice genetics problem suitable for an open-book exam (i.e. not based on memorization) or to answer and critique at least two problems previously created by other students.
I had used Peerwise a couple of years ago in a first-year class. Each student only had to create one question and critique four. I had found their questions to be full of confusions, poorly written and lacking important information, but the critiques were quite good.
This time, my students have told me that they don't mind answering the problems other students have posed but find creating their own to be quite difficult. The first problems I had looked at hadn't been very good. But last week I started going through their more recent problems, looking for ideas I might use in creating problems for the upcoming midterm, and I was pleasantly surprised by the high quality of many of the problems they had developed.
I was so impressed by the students' questions that I decided to use them for the midterm, rather than writing my own. I downloaded a range of questions to consider - some of them were very good but much too difficult for a 45-minute midterm. I had thought that the good questions might have all been written by one or a few students, but when I checked their usernames I found that 16 of the 17 questions had been written by different students; this means that most and maybe all students are creating good problems!
I used 14 questions for the exam - most of them I was able to leave unchanged, but some required minor editing for clarity. The students must have recognized their own questions, but I haven't seen anything on the course discussion board to suggest that they've realized that they collectively created all the questions on the exam. I'll tell them on Monday.
Thursday, February 24, 2011
The CPR site counts html tags as words...
The instructions for my Calibrated Peer Review assignment specified that the reports should be between 300 and 500 words long. And a nice feature of the CPR assignment setup is that it polices this - the instructor specifies the acceptable word range for submissions, and submissions that are too short or too long are not accepted. But a problem has arisen because the CPR submission form only accepts plain text, and needs HTML tags even for paragraph breaks.
Of course very few of my students will be familiar with HTML tags (it's a bit shocking that I'm more web-savvy than most of them). I thought I had solved the formatting problem by giving them a link to a web page where they could paste in their formatted text (from a Word document) and have it converted to HTML, which they could then paste into the CPR submission form.
But the CPR form counts the HTML tags as words and, for students who have meticulously formatted their reports in Word, this adds hundreds of pseudo-words that push it way over the word limit.
So I've raised the CPR word limit to 1000, but warned students that their final submission (after CPR is complete) will still have to be no more than 500 words.
Of course very few of my students will be familiar with HTML tags (it's a bit shocking that I'm more web-savvy than most of them). I thought I had solved the formatting problem by giving them a link to a web page where they could paste in their formatted text (from a Word document) and have it converted to HTML, which they could then paste into the CPR submission form.
But the CPR form counts the HTML tags as words and, for students who have meticulously formatted their reports in Word, this adds hundreds of pseudo-words that push it way over the word limit.
So I've raised the CPR word limit to 1000, but warned students that their final submission (after CPR is complete) will still have to be no more than 500 words.
Friday, February 18, 2011
CPR almost ready to go, surpasses all expectations
The Calibrated Peer Review assignment for my genetics class is almost ready to go, thanks to lots of work by our Faculty of Science technical support person working with the Centre for Learning, Teaching and Technology (or is it 'Teaching, Learning and Technology...?).
The legal indemnification issue has been resolved; apparently it arose due to a misunderstanding between the two parties about which university was meant by the phrase "the University". The software has been installed into UBC's elearning system, and the course has been created.
Setting up the student accounts turned out to be a bit of a hassle. The local CPR setup needs to be told, for each course, the names and IDs of the students who are entitled to create user profiles. In principle this info should be uploadable from the class list for the course, but for some reason it had to be entered manually (by the tech person, not me). This problem will need to be resolved by September, as we'll have >400 students in the course then. (Note added later: In my Instructor view I see a page for uploading students from a class list file; I don't know if this is what wouldn't work for the tech. Later: this problem has been solved.)
The students have been sent emails telling them to set up their user profiles (just password, email address and secret question/answer) and complete the introductory tutorial and pretest, and several have already done so. Once they have an account they must agree to the CPR Terms of Use; this requires them to promise to abide by UCLA's Student Conduct Code! (I couldn't check out the code because the link to it is broken.)
The tutorial and pretest are an excellent feature. The tutorial takes students through ten pages of information about the different steps of a CPR assignment. The pages are very clear, with good diagrams and illustrative screen shots. Once the student has been through the tutorial they take a pretest to check that they've understood how the CPR process works, with 12 Yes/No questions. The pretest isn't graded, but the system records that the student has completed it.
At this point students are ready to begin their assignment. Unfortunately mine can't do this yet, because a setup problem won't let me import the assignment information from its home on the CPR Central website at UCLA. I've finished creating all the assignment components (learning objectives, instructions for the students, calibration essays, review questions, answers and feedback to review questions for each calibration essay). The local CPR interface asks me if I want to activate a new assignment, and then asks for my CPR Central userID and password so it can connect to the assignments I have there. But it can't make the connection.
The support tech had her CTLT tech working on this problem yesterday (Later: they're waiting to hear back from the UCLA tech). If it's solved today (and no other bugs surface) I'll be able to tell the students to submit their drafts by the Sunday midnight deadline. If it persists I'll have to extend the submission deadline again, and push back the dates for completing the various stages of the assignment. Fortunately I set up the original dates to have the final polished submission due several weeks before the end of term, so moving to a later deadline won't be a big problem.
Over all I'm very impressed with the high quality of the CPR resources and interface. Every step has been very straightforward and easy to understand, and the supporting materials for both instructors and students are very well designed. (Of course, the technical people may think differently...)
The legal indemnification issue has been resolved; apparently it arose due to a misunderstanding between the two parties about which university was meant by the phrase "the University". The software has been installed into UBC's elearning system, and the course has been created.
Setting up the student accounts turned out to be a bit of a hassle. The local CPR setup needs to be told, for each course, the names and IDs of the students who are entitled to create user profiles. In principle this info should be uploadable from the class list for the course, but for some reason it had to be entered manually (by the tech person, not me). This problem will need to be resolved by September, as we'll have >400 students in the course then. (Note added later: In my Instructor view I see a page for uploading students from a class list file; I don't know if this is what wouldn't work for the tech. Later: this problem has been solved.)
The students have been sent emails telling them to set up their user profiles (just password, email address and secret question/answer) and complete the introductory tutorial and pretest, and several have already done so. Once they have an account they must agree to the CPR Terms of Use; this requires them to promise to abide by UCLA's Student Conduct Code! (I couldn't check out the code because the link to it is broken.)
The tutorial and pretest are an excellent feature. The tutorial takes students through ten pages of information about the different steps of a CPR assignment. The pages are very clear, with good diagrams and illustrative screen shots. Once the student has been through the tutorial they take a pretest to check that they've understood how the CPR process works, with 12 Yes/No questions. The pretest isn't graded, but the system records that the student has completed it.
At this point students are ready to begin their assignment. Unfortunately mine can't do this yet, because a setup problem won't let me import the assignment information from its home on the CPR Central website at UCLA. I've finished creating all the assignment components (learning objectives, instructions for the students, calibration essays, review questions, answers and feedback to review questions for each calibration essay). The local CPR interface asks me if I want to activate a new assignment, and then asks for my CPR Central userID and password so it can connect to the assignments I have there. But it can't make the connection.
The support tech had her CTLT tech working on this problem yesterday (Later: they're waiting to hear back from the UCLA tech). If it's solved today (and no other bugs surface) I'll be able to tell the students to submit their drafts by the Sunday midnight deadline. If it persists I'll have to extend the submission deadline again, and push back the dates for completing the various stages of the assignment. Fortunately I set up the original dates to have the final polished submission due several weeks before the end of term, so moving to a later deadline won't be a big problem.
Over all I'm very impressed with the high quality of the CPR resources and interface. Every step has been very straightforward and easy to understand, and the supporting materials for both instructors and students are very well designed. (Of course, the technical people may think differently...)
Tuesday, February 15, 2011
email to a textbook rep
Dear textbook rep,
I haven't received the brand-new textbook yet (author's name and title redacted), but today I did get the Instructor's Media CD for it, and I've been through all the sets of slides. I'm afraid it's not at all what we're looking for.
Because the title promises a genomics approach, I was hoping for a text that presented the basic principles of genetics in the context of our new spectacular information about human and other genomes. Instead it's yet another old-fashioned Genetic Analysis textbook, with no modern genomics at all!
The first chapter covers molecular biology at a high-school senior/Intro Bio level, and is followed by the standard four chapters teaching classical genetics (Mendel and single-gene inheritance, mitosis and meiosis, linkage and mapping, chromosome structure and behaviour. All the same material that's been taught since the 1960s, with the odd snippet about gene function, such as the molecular defect in Mendel's wrinkled pea allele. Then a series of more molecular chapters (DNA replication, bacteria and their viruses, gene expression, gene regulation), all still very classical in the information they present.
Chapter 10 claims to be about genomics. But what does it contain? The same old molecular cloning and genetic engineering methods, supplemented with explanations about how microarrays work and how a germ-line transformation is done. The only genomics is the genome of the bacterium Mycoplasma genitalium, published 15 years ago! No human examples at all, just flies and fish and rice.
Then more standard chapters with the standard material present in every other textbook: development, mutation and DNA repair, cell cycles and cancer, classical population genetics, classical quantitative genetics.
Students taking an introductory genetics course don't need to learn how to clone genes, they don't need to know what Mendel did, and they certainly don't need to understand how a Southern blot is done. Even professional geneticists never do Southern blots any more!
More than anything students need to understand their own genomes. They need to know how inheritance works, and how genes affect phenotypes. They need to understand natural genetic variation, in their own and other species. These concepts aren't particularly difficult, except maybe when they're embedded in the baggage of classical genetic analysis.
I'm quite disappointed, as I was hoping that this would finally be a textbook we could use. But I guess I'll continue to make do without any assigned textbook.
I'm attaching a copy of the lecture schedule for this course, just in case you know of any other textbook that might take a more modern approach.
Rosie
I haven't received the brand-new textbook yet (author's name and title redacted), but today I did get the Instructor's Media CD for it, and I've been through all the sets of slides. I'm afraid it's not at all what we're looking for.
Because the title promises a genomics approach, I was hoping for a text that presented the basic principles of genetics in the context of our new spectacular information about human and other genomes. Instead it's yet another old-fashioned Genetic Analysis textbook, with no modern genomics at all!
The first chapter covers molecular biology at a high-school senior/Intro Bio level, and is followed by the standard four chapters teaching classical genetics (Mendel and single-gene inheritance, mitosis and meiosis, linkage and mapping, chromosome structure and behaviour. All the same material that's been taught since the 1960s, with the odd snippet about gene function, such as the molecular defect in Mendel's wrinkled pea allele. Then a series of more molecular chapters (DNA replication, bacteria and their viruses, gene expression, gene regulation), all still very classical in the information they present.
Chapter 10 claims to be about genomics. But what does it contain? The same old molecular cloning and genetic engineering methods, supplemented with explanations about how microarrays work and how a germ-line transformation is done. The only genomics is the genome of the bacterium Mycoplasma genitalium, published 15 years ago! No human examples at all, just flies and fish and rice.
Then more standard chapters with the standard material present in every other textbook: development, mutation and DNA repair, cell cycles and cancer, classical population genetics, classical quantitative genetics.
Students taking an introductory genetics course don't need to learn how to clone genes, they don't need to know what Mendel did, and they certainly don't need to understand how a Southern blot is done. Even professional geneticists never do Southern blots any more!
More than anything students need to understand their own genomes. They need to know how inheritance works, and how genes affect phenotypes. They need to understand natural genetic variation, in their own and other species. These concepts aren't particularly difficult, except maybe when they're embedded in the baggage of classical genetic analysis.
I'm quite disappointed, as I was hoping that this would finally be a textbook we could use. But I guess I'll continue to make do without any assigned textbook.
I'm attaching a copy of the lecture schedule for this course, just in case you know of any other textbook that might take a more modern approach.
Rosie
Sunday, February 13, 2011
Comparing the midterm results
Here's a graph comparing how each student in my pilot genetics course did on the first and second mini-midterms. The first tested their understanding of the relationship between genotype and phenotype, especially in diploid organisms. The second tested their understanding of how alleles are inherited through meiosis and mating. The material on the two tests did not overlap at all. Both were 25-minute, open book, and mostly multiple choice and short answer.
The blue dashed line separates students who did better on the first midterm (dots above the line) from students who did better on the second (dots below the line). Although many students did worse on the second one, 14 of the 38 did better.
The two dots in the lower left square are students who failed both tests. The single dot in the upper-left pink square is the student who failed the first test but passed the second. The ten dots in the lower-right pink square are students who all passed the first midterm but failed the second (some very badly).
We've just entered the marks for each question, but I'm starting to think that the dataset is too small to allow any more useful generalizations.
The two dots in the lower left square are students who failed both tests. The single dot in the upper-left pink square is the student who failed the first test but passed the second. The ten dots in the lower-right pink square are students who all passed the first midterm but failed the second (some very badly).
We've just entered the marks for each question, but I'm starting to think that the dataset is too small to allow any more useful generalizations.
Saturday, February 12, 2011
Interpreting a grade distribution
Yesterday my genetics students wrote their second 'mini-midterm'. This was a 25-minute quiz on the material we've covered in the last two weeks. Everything went smoothly, the papers are graded, and the grades are posted along with the answer key. But I don't know how to interpret the grade distribution. Here's the histogram:
The quiz was open book, with five questions that were designed to require some thoughtful analysis but be very easy to mark (grading 38 papers took four of us about 30 minutes). The questions weren't too difficult - 3 students earned perfect scores, and most had finished before the time was up. They also weren't too easy - 12 students failed, and the mean score was only 15.8/25 (62%).
We expect grades on most tests to give a 'normal' distribution - the bell-shaped curve. The curve may be skewed to the right if the exam was too easy, or to the left if it was too hard. A bimodal curve (with two humps) usually means that the students fall into two groups - those who have acquired some key skills and those who haven't.
But the grade distribution for this quiz looks flat to me, not really a curve at all. It's flat all the way from 12% to 100%, with no more scores close to the mean than elsewhere. I've never seen a grade distribution like this before and I don't know how it should be interpreted. I can find technical descriptions of this shape in the context of a normal distribution (it's 'platykurtic') but I can't find any consideration of what this would imply about either students' abilities or the design of the test.
Maybe I'll ask my colleague in Curriculum Studies...
The quiz was open book, with five questions that were designed to require some thoughtful analysis but be very easy to mark (grading 38 papers took four of us about 30 minutes). The questions weren't too difficult - 3 students earned perfect scores, and most had finished before the time was up. They also weren't too easy - 12 students failed, and the mean score was only 15.8/25 (62%).
We expect grades on most tests to give a 'normal' distribution - the bell-shaped curve. The curve may be skewed to the right if the exam was too easy, or to the left if it was too hard. A bimodal curve (with two humps) usually means that the students fall into two groups - those who have acquired some key skills and those who haven't.
But the grade distribution for this quiz looks flat to me, not really a curve at all. It's flat all the way from 12% to 100%, with no more scores close to the mean than elsewhere. I've never seen a grade distribution like this before and I don't know how it should be interpreted. I can find technical descriptions of this shape in the context of a normal distribution (it's 'platykurtic') but I can't find any consideration of what this would imply about either students' abilities or the design of the test.
Maybe I'll ask my colleague in Curriculum Studies...
CPR update
The original plan for our Calibrated Peer Review assignment was that students would submit their draft assignments by last night. Next week is Reading Week, a mid-term break with no classes or assignments due, so this would give them have a week off before doing their three calibration reviews (one week) and their reviews of three student submissions (the next week).
But getting the CPR system up and running at UBC is taking longer than I had hoped, so I've set back the due date until next Sunday (at the end of Reading Week).
I had anticipated that there might be problems't getting the CPR software to work as intended, and had set the original the due date with the plan that it could be changed. However the problem isn't software implementation at all, but the legal license agreement between UBC and UCLA.
The agreement includes an 'indemnity' clause that (I think) protects UCLA from the consequences of anything bad that UBC might do. But UBC's lawyers have provided a standard indemnification clause that isn't as sweeping as the one UCLA specifies. Any changes to this could require months (years?) of back-and-forth between the legal counsels for the two institutions.
The new version of CPR (CPR4) is the first one to have a local component installed on the user's computers - the previous versions all ran entirely on the UCLA system and were remotely accessed by students at other institutions. So I was concerned that UBC might be the first foreign user, and that all the legal bugs still had to be worked out.
Luckily my UCLA contact assures me that there should be no legal problems, so the local installation is proceeding. With luck, we'll get all the bugs out next week and be ready for the students next weekend.
But getting the CPR system up and running at UBC is taking longer than I had hoped, so I've set back the due date until next Sunday (at the end of Reading Week).
I had anticipated that there might be problems't getting the CPR software to work as intended, and had set the original the due date with the plan that it could be changed. However the problem isn't software implementation at all, but the legal license agreement between UBC and UCLA.
The agreement includes an 'indemnity' clause that (I think) protects UCLA from the consequences of anything bad that UBC might do. But UBC's lawyers have provided a standard indemnification clause that isn't as sweeping as the one UCLA specifies. Any changes to this could require months (years?) of back-and-forth between the legal counsels for the two institutions.
The new version of CPR (CPR4) is the first one to have a local component installed on the user's computers - the previous versions all ran entirely on the UCLA system and were remotely accessed by students at other institutions. So I was concerned that UBC might be the first foreign user, and that all the legal bugs still had to be worked out.
Luckily my UCLA contact assures me that there should be no legal problems, so the local installation is proceeding. With luck, we'll get all the bugs out next week and be ready for the students next weekend.
Wednesday, February 09, 2011
Saturday, February 05, 2011
Calibrated Peer Review is 'developmental' for instructors as well as students
I've set aside the original plan of having the students in my new genetics course write letters to the editor about genetics reporting errors, because finding suitable errors turned out to be too difficult for them (but thanks for the suggestions). The new plan (they voted and chose it) is that they will instead write short reports titled 'My Favourite Human SNP'. This looks like it will work well both for this small pilot class and for the ~500 students we expect in September, because the pool of SNPs with associated phenotype information is large and growing fast. The assignment still has the benefit of letting each student choose their own topic, and of being very suitable for Calibrated Peer Review (CPR).
I've already posted on our course-management system a page of instructions to the students about what is expected in this assignment, but now I'm creating the assignment within the CPR system. Although the CPR interface for the students is run locally (i.e. at UBC) under the new CPR4 system, assignments are generated centrally on the CPRCentral server at UCLA. This allows the central server to maintain consistency (all authors have to use the same structure) and to provide a library of past assignments that any instructor may use or adapt.
In principle I could have adapted one of the many existing assignments from the CPR library, but none of them were suitable. Instead I'm writing my own from scratch, using some of the library assignments as models. Now I'm working through the surprisingly many steps of creating an assignment from scratch, using the detailed Authoring Guide that CPR provides. This turns out to be much less daunting than I had expected, and much more enjoyable and educational (for me).
The first step is choosing a title and descriptive information for the assignment. Because all assignments are put it the open library for others to later adapt, it's more important that this title be informative for future users than that it be the best title for the students. A suggested student title can be included in the explanatory notes. The assignment is also given a topic area (mine is Biology - Genetics), keywords, and a user level (mine is Lower-division undergraduate). This information is used by other instructors but I don't think it's seen by the students.
The next step is writing explicit Learning Goals for the assignment. I hadn't done this for an assignment before, so thinking through what I wanted the students to learn (guidelines are provided) helped to educate me about the value of setting such goals as well as providing the students with clear expectations (students see these at the top of the assignment page).
Writing the Learning Goals was the first place where the lack of formatting power raised its ugly head. The CPR interface accepts only plain text with or without html codes (e.g. you have to manually insert
wherever you want a line break). I expected this to be very frustrating but the combination of a nice page of html tags and my kindergarden-level html skills let me format lists of points and boldface headings without a hitch. However I anticipate that most of the students will have more difficulty with this - at a minimum they'll have to put in the line breaks. (Yes, I know paragraph breaks are better, but if you don't have any idea how html tags work, line breaks are easier to understand.) I'll need to create a short example page for them on Vista showing how to do this. They can also just use a web page I've found that lets them paste their Word-generated text into a box that converts it to HTML.
The next steps create the resources the students should use to carry out their assignment (to generate the reports that will be assessed by their peers).
I. Guidance for Studying Source Materials: This tells students how they should gather the information and understanding that will go into their report. It has two parts, some text describing the source materials (handouts, textbook, articles, etc) and then some hyperlinks to web pages. Here I just used modified text from the first part of the instructions I'd already posted for my students. I didn't initially notice the second part of this section, so I hand-coded the hyperlinks into my text section - that worked fine.
II. Guidance for Writing Your Text: This tells students what their reports should say and how this should be presented. Here I used modified text from the second part of my posted instructions.
III. A 'writing prompt'. This appears above the text entry box and gives the student specific instructions about format and text entry.
Now comes the big task of preparing the 'calibration essays'. Three of these are required; each student will evaluate these using the series of rubric questions that you create in the next step. One calibration essay is tagged 'high quality', one 'middle quality', and one 'low quality', but they can differ along several axes, with the caution that students will have a hard time evaluating the content of an essay with too many writing errors. I had already decided on the SNPs I would write these about, but they're only about half done right now so I'll write more about these later.
The next step is creating the series of questions that students will use to evaluate the essays. The interface makes this quite easy; it lets you specify what kind of answer is expected (yes/no, none/some/many, A/B/C (where you specify what A, B and C mean), and whether the student is expected to enter some text in a box. I had some questions in mind from the original instructions I'd given the students, and I thought of more while I was working on the calibration essays. Now I have 17 questions, which I suspect may be too many, even though most of them are very simple: "Does the report contain spelling errors? (none/some/many)"; "Does the report say how common the phenotype of interest is? (yes/no)". I'll no doubt refine these once I'm into the next step.
The final big step is, for each evaluation question, designating the correct answer for each of the three calibration reports and writing a brief explanation of why this is the correct choice in each case. This is going to take a while, and I can't begin until I finish writing the calibration essays. But I'm looking forward to it, because I can see how valuable it will be.
So the above is a lot of details - what's the big picture? UBC's learning technology people agreed to set up this CPR trial because they saw CPR as much more 'developmental' for the students than our existing peer-review options (iPeer and the peer review component of Turnitin). I agree, but I'm finding that the experience of creating the assignment is also developmental for me - I'm being gently led through a series of steps that greatly improve the learning experience I'm providing for my students, with instruction at each step so I see both why the step is valuable and how best to implement it.
I've already posted on our course-management system a page of instructions to the students about what is expected in this assignment, but now I'm creating the assignment within the CPR system. Although the CPR interface for the students is run locally (i.e. at UBC) under the new CPR4 system, assignments are generated centrally on the CPRCentral server at UCLA. This allows the central server to maintain consistency (all authors have to use the same structure) and to provide a library of past assignments that any instructor may use or adapt.
In principle I could have adapted one of the many existing assignments from the CPR library, but none of them were suitable. Instead I'm writing my own from scratch, using some of the library assignments as models. Now I'm working through the surprisingly many steps of creating an assignment from scratch, using the detailed Authoring Guide that CPR provides. This turns out to be much less daunting than I had expected, and much more enjoyable and educational (for me).
The first step is choosing a title and descriptive information for the assignment. Because all assignments are put it the open library for others to later adapt, it's more important that this title be informative for future users than that it be the best title for the students. A suggested student title can be included in the explanatory notes. The assignment is also given a topic area (mine is Biology - Genetics), keywords, and a user level (mine is Lower-division undergraduate). This information is used by other instructors but I don't think it's seen by the students.
The next step is writing explicit Learning Goals for the assignment. I hadn't done this for an assignment before, so thinking through what I wanted the students to learn (guidelines are provided) helped to educate me about the value of setting such goals as well as providing the students with clear expectations (students see these at the top of the assignment page).
Writing the Learning Goals was the first place where the lack of formatting power raised its ugly head. The CPR interface accepts only plain text with or without html codes (e.g. you have to manually insert
wherever you want a line break). I expected this to be very frustrating but the combination of a nice page of html tags and my kindergarden-level html skills let me format lists of points and boldface headings without a hitch. However I anticipate that most of the students will have more difficulty with this - at a minimum they'll have to put in the line breaks. (Yes, I know paragraph breaks are better, but if you don't have any idea how html tags work, line breaks are easier to understand.) I'll need to create a short example page for them on Vista showing how to do this. They can also just use a web page I've found that lets them paste their Word-generated text into a box that converts it to HTML.
The next steps create the resources the students should use to carry out their assignment (to generate the reports that will be assessed by their peers).
I. Guidance for Studying Source Materials: This tells students how they should gather the information and understanding that will go into their report. It has two parts, some text describing the source materials (handouts, textbook, articles, etc) and then some hyperlinks to web pages. Here I just used modified text from the first part of the instructions I'd already posted for my students. I didn't initially notice the second part of this section, so I hand-coded the hyperlinks into my text section - that worked fine.
II. Guidance for Writing Your Text: This tells students what their reports should say and how this should be presented. Here I used modified text from the second part of my posted instructions.
III. A 'writing prompt'. This appears above the text entry box and gives the student specific instructions about format and text entry.
Now comes the big task of preparing the 'calibration essays'. Three of these are required; each student will evaluate these using the series of rubric questions that you create in the next step. One calibration essay is tagged 'high quality', one 'middle quality', and one 'low quality', but they can differ along several axes, with the caution that students will have a hard time evaluating the content of an essay with too many writing errors. I had already decided on the SNPs I would write these about, but they're only about half done right now so I'll write more about these later.
The next step is creating the series of questions that students will use to evaluate the essays. The interface makes this quite easy; it lets you specify what kind of answer is expected (yes/no, none/some/many, A/B/C (where you specify what A, B and C mean), and whether the student is expected to enter some text in a box. I had some questions in mind from the original instructions I'd given the students, and I thought of more while I was working on the calibration essays. Now I have 17 questions, which I suspect may be too many, even though most of them are very simple: "Does the report contain spelling errors? (none/some/many)"; "Does the report say how common the phenotype of interest is? (yes/no)". I'll no doubt refine these once I'm into the next step.
The final big step is, for each evaluation question, designating the correct answer for each of the three calibration reports and writing a brief explanation of why this is the correct choice in each case. This is going to take a while, and I can't begin until I finish writing the calibration essays. But I'm looking forward to it, because I can see how valuable it will be.
So the above is a lot of details - what's the big picture? UBC's learning technology people agreed to set up this CPR trial because they saw CPR as much more 'developmental' for the students than our existing peer-review options (iPeer and the peer review component of Turnitin). I agree, but I'm finding that the experience of creating the assignment is also developmental for me - I'm being gently led through a series of steps that greatly improve the learning experience I'm providing for my students, with instruction at each step so I see both why the step is valuable and how best to implement it.
Saturday, January 29, 2011
Looking for examples of BAD genetics journalism
The students in my introductory genetics class have an unusual assignment - each of them has to find an error in the reporting of genetics and write a letter to the editor about it. They're not very skilled at finding examples of such errors (and I'm afraid I haven't given them much time), so I'm asking the twitterverse for help. If you've recently wrung your hands about some egregious error in the reporting of some advance in DNA or genetics research, we'd be very grateful if you would post a link (or other identifying info) in the comments.
Here's some more information about the assignment:
The students are asked to find somewhere in the media where an incorrect statement is made about a genetic topic. This could be in a tabloid, newspaper or magazine, on television, or in a news-media online source. (General blog posts are not eligible, though media-affiliated ones are.)
These students are only part-way through their first genetics course, so the error needs to be pretty basic. Examples I've given them include
The present class is only 40 students, but if this assignment works well (and the CPR works well) we'd like to run it for 500 students next Fall. This would lead to a barrage of letters to the editor complaining about the poor quality of their genetics coverage, and might even lead to an improvement in future reporting.
Here's some more information about the assignment:
The students are asked to find somewhere in the media where an incorrect statement is made about a genetic topic. This could be in a tabloid, newspaper or magazine, on television, or in a news-media online source. (General blog posts are not eligible, though media-affiliated ones are.)
These students are only part-way through their first genetics course, so the error needs to be pretty basic. Examples I've given them include
- describing genome sequencing as 'cracking the genetic code'
- describing a bacterium with arsenic in its DNA as 'a new form of life'
- credulously reporting about the predicted effect of what turns out to be an imaginary gene
- claiming that gene A causes behaviour B, when it only slightly increases the probability of the behaviour.
The present class is only 40 students, but if this assignment works well (and the CPR works well) we'd like to run it for 500 students next Fall. This would lead to a barrage of letters to the editor complaining about the poor quality of their genetics coverage, and might even lead to an improvement in future reporting.
Thursday, January 27, 2011
How we define the phenotype is critical
The post-doc and I are discussing the following genetics problem, taken from a textbook:
What I originally said:
What I say now (after quite a bit of thinking):
First, we're told that the mutant allele is a null allele, so the heterozygote is expected to have half the normal amount of protein (5 units instead of 10) Since 8 units are needed for the normal phenotype, the mutant heterozygote will not be normal. So the mutant allele certainly is not recessive.
(Here I'm assuming that the defect in one allele doesn't cause the other allele to be upregulated. That's a possible answer to part b, though I doubt it was what the questioner was looking for, since this question comes from the first chapter on simple Mendelian inheritance.)
We're not told the phenotype of a mutant homozygote, so before considering whether the mutant allele could be dominant to the wildtype allele we need to carefully identify the phenotype in question. The term 'phenotype' can have different meanings even for a given pair of alleles, depending on what is being observed and how it is being categorized.
For example, if a pigment is being observed it could be treated qualitatively (red/white; red/pink/white; present/absent) or quantitatively (how much pigment is present). Phenotypes are usually treated qualitatively in genetics textbooks, with quantitative phenotypes segregated into a special chapter. But most real phenotypes have gradations, and the observer must decide whether to treat them qualitatively (with 2, 3 or more categories) or qantitatively.
Qualitative categories are usually chosen to reflect the underlying genetic effects. For example, an observer might initially categorize flower pigment as red/white, and later realize that the 'red' category should be divided into 'red' and 'pink' because this better explained how the colours were being inherited. If a gene were later discovered that modulated pigment production, the observer might then treat pigment quantitatively.
The problem posed above doesn't give us explicit guidance about whether this phenotype should be treated qualitatively or quantitatively. Normal is presented as an ordinary word, not flagged as a special term by quotes or italicization, so we could certainly interpret it quantitatively. However it could be meant qualitatively, although we're not given any clues to what the categories would be (normal/abnormal? normal/abnormal/severely abnormal?).
If the phenotype is to be treated quantitatively (with 'normal' just taking its ordinary English meaning), then the mutant homozygote is expected to have a more severe abnormality than the heterozygote, so the allele would not be dominant.
But the postdoc argues that it's just as reasonable to treat the phenotype qualitatively with two categories, 'normal' and 'abnormal', and I agree that under this definition the mutant allele would be considered dominant.
However I think that requiring the phenotype to be defined this way is tantamount to making this a 'trick question', because this definition implies that the person posing the question deliberately ignored whatever information might be given by a more nuanced definition (one that considered possible differences between the mutant homozygote and the heterozygote).
Because the wording of the question doesn't favour this interpretation over any other, we should go for interpretations that are more reasonable - qualitative with more than two categories, or quantitative.
Would it be OK to say that the mutant is dominant because the heterozygote and mutant homozygote really do have identical phenotypes under more nuanced definitions (i.e. that they are equally abnormal)? No, because this would require a biologically unreasonable explanation for the dominance - either the null allele in the heterozygote must completely prevent expression of the normal allele, or the presence of two null alleles in the homozygote must allow them to produce 5 units. The former is very unlikely though not impossible, and the latter is inconsistent with the meaning of 'null allele'.
Later: I've heard back from the person who wrote this question. He indeed meant 'dominant' rather than 'mutant, and his intended answer agrees with that of the postdoc - that all non-normal phenotypes should be lumped together into the 'abnormal' category, which would make the null allele 'dominant'.
I think this is both scientifically bizarre and pedagogically misleading. It reinforces the erroneous assumption that alleles must be either dominant or recessive, and requires a very improbable explanation to be treated as typical. Either question a should give a third option (recessive, dominant or neither) or the question should be framed with "What is wrong with this question?". Question b can be deleted.
Q. For a certain gene in a diploid organism, eight units of protein product are needed for normal function. Each wild-type allele produces five units.
a. If a mutation creates a null allele, do you think this allele will be recessive or mutant*?
b. What assumptions need to be made to answer part a?*Note: I don't know what the word 'mutant' means here, since we already know that the allele is mutant. I suspect it's an error so I initially ignored it.
What I originally said:
The mutation is not recessive to the wildtype allele, because the heterozygote has a different phenotype than the wildtype homozygote. I don't think this conclusion requires any assumptions other than the usual definition of recessive.However the postdoc and others have been arguing that mutation should be interpreted as dominant. This requires interpreting the word 'mutant' as an error where 'dominant' was meant, which is not unreasonable.
What I say now (after quite a bit of thinking):
First, we're told that the mutant allele is a null allele, so the heterozygote is expected to have half the normal amount of protein (5 units instead of 10) Since 8 units are needed for the normal phenotype, the mutant heterozygote will not be normal. So the mutant allele certainly is not recessive.
(Here I'm assuming that the defect in one allele doesn't cause the other allele to be upregulated. That's a possible answer to part b, though I doubt it was what the questioner was looking for, since this question comes from the first chapter on simple Mendelian inheritance.)
We're not told the phenotype of a mutant homozygote, so before considering whether the mutant allele could be dominant to the wildtype allele we need to carefully identify the phenotype in question. The term 'phenotype' can have different meanings even for a given pair of alleles, depending on what is being observed and how it is being categorized.
For example, if a pigment is being observed it could be treated qualitatively (red/white; red/pink/white; present/absent) or quantitatively (how much pigment is present). Phenotypes are usually treated qualitatively in genetics textbooks, with quantitative phenotypes segregated into a special chapter. But most real phenotypes have gradations, and the observer must decide whether to treat them qualitatively (with 2, 3 or more categories) or qantitatively.
Qualitative categories are usually chosen to reflect the underlying genetic effects. For example, an observer might initially categorize flower pigment as red/white, and later realize that the 'red' category should be divided into 'red' and 'pink' because this better explained how the colours were being inherited. If a gene were later discovered that modulated pigment production, the observer might then treat pigment quantitatively.
The problem posed above doesn't give us explicit guidance about whether this phenotype should be treated qualitatively or quantitatively. Normal is presented as an ordinary word, not flagged as a special term by quotes or italicization, so we could certainly interpret it quantitatively. However it could be meant qualitatively, although we're not given any clues to what the categories would be (normal/abnormal? normal/abnormal/severely abnormal?).
If the phenotype is to be treated quantitatively (with 'normal' just taking its ordinary English meaning), then the mutant homozygote is expected to have a more severe abnormality than the heterozygote, so the allele would not be dominant.
But the postdoc argues that it's just as reasonable to treat the phenotype qualitatively with two categories, 'normal' and 'abnormal', and I agree that under this definition the mutant allele would be considered dominant.
However I think that requiring the phenotype to be defined this way is tantamount to making this a 'trick question', because this definition implies that the person posing the question deliberately ignored whatever information might be given by a more nuanced definition (one that considered possible differences between the mutant homozygote and the heterozygote).
Because the wording of the question doesn't favour this interpretation over any other, we should go for interpretations that are more reasonable - qualitative with more than two categories, or quantitative.
Would it be OK to say that the mutant is dominant because the heterozygote and mutant homozygote really do have identical phenotypes under more nuanced definitions (i.e. that they are equally abnormal)? No, because this would require a biologically unreasonable explanation for the dominance - either the null allele in the heterozygote must completely prevent expression of the normal allele, or the presence of two null alleles in the homozygote must allow them to produce 5 units. The former is very unlikely though not impossible, and the latter is inconsistent with the meaning of 'null allele'.
Later: I've heard back from the person who wrote this question. He indeed meant 'dominant' rather than 'mutant, and his intended answer agrees with that of the postdoc - that all non-normal phenotypes should be lumped together into the 'abnormal' category, which would make the null allele 'dominant'.
I think this is both scientifically bizarre and pedagogically misleading. It reinforces the erroneous assumption that alleles must be either dominant or recessive, and requires a very improbable explanation to be treated as typical. Either question a should give a third option (recessive, dominant or neither) or the question should be framed with "What is wrong with this question?". Question b can be deleted.
Tuesday, January 25, 2011
Answering a complex problem after discussing it in tutorial
My genetics students are complaining about the way I've designed the tutorials. I have them spend the first part of each two-hour tutorial in a structured discussion of the topics covered by the past week's classes, and the second half working on a complex genetics problem. It's this second part that's generating the complaints.
They're given the problem in advance, and are asked to print it out and make a preliminary attempt at it before tutorial. I've told them that this attempt can be quite superficial; it's only worth 1 point (out of 5). They turn in this attempt at the start of tutorial, and are given a blank copy of the problem to work on. The students then work on the problem in groups of 3-4 at the chalkboards. (This classroom is in the old math building so it has lovely chalkboards filling three walls.) Different groups then explain to the class their suggested answers to the different parts of the problem, and students discuss these answers. They also discuss how the problem might be adapted or modified for use in different settings, for example, changing the organism so it can be reused on a test, or making part of it into a shorter stand-alone problem. (In future we'll try to get them to also explicitly discuss what is needed for a good written answer to the problem, but they're not ready for that yet.)
All this seems to be OK with them. But the final step is for each student to write out a careful answer to the problem they've been discussing, as if this was an exam setting. These answers are handed in and marked; they're worth 4 points. At present the group work is left on the chalkboards while students are writing their answers, but I've told them that in a few weeks we'll start erasing the boards before they write their answers.
Students are complaining that this is a waste of their time, that they don't learn anything by having to write answers after they've already seen how the problem should be answered, and that they would learn more by spending the time in additional discussion. I disagree - I think that observing the right answer doesn't lead to much learning, and that having to apply what they've just observed by creating a written answer adds a lot.
In tomorrow's lecture I'm going to show them some data that might help them see the value in this. It's from a paper that just appeared in Science (Karpicke and Blunt). In both of the two studies they describe, the authors had students spend 5 minutes reading a half-page of text about a biological topic, and then consolidate what they'd read in various ways. The students were then asked to predict how much they would remember a week later. A week later they were tested on each topic.
In the first study the students either (i) did nothing more, (ii) reread the text three more times, (iii) spent 25 minutes making a concept map with the text, or (iv) tested their recall immediately by writing about it for 10 minutes, then reread the text, and retested their recall. In the second study the students either (v) spent 25 minutes making a concept map or (vi) tested their recall, reread the text, and retested their recall. In the first study each student read only one text and was tested a week later with a short-answer test. In the second each student was given two texts, one learned with a concept map and one with recall testing, and these were tested a week later using either a short-answer test or a concept map (in randomized combinations).
In both studies the students predicted that they'd remember more with the non-testing methods, but in the post-tests they always scored substantially higher when they had consolidated their reading by testing their recall. Here are edited versions of their graphs:
I'm going to show my students this study in tomorrow's lecture, and I'm going to give them two conclusions: First, people are not very good judges of how much they've learned. (So my students should realize that their opinions of how much they learn by different tutorial activities may well be mistaken.) Second, testing oneself is an excellent way to learn. (So my students should realize that having to develop a written answer after a discussion is a valuable way to reinforce what they've discussed.)
This will take a few minutes that I could otherwise spend talking about mitosis but I think learning how to learn is more important. The students have a mini-midterm coming up on Friday, so they should be fairly receptive to ideas about how to learn. I don't expect that this new data will convince them all that my tutorial design is good (that's why I wrote 'should' above instead of 'will') but at least they'll realize that I'm not just doing it to to be mean.
They're given the problem in advance, and are asked to print it out and make a preliminary attempt at it before tutorial. I've told them that this attempt can be quite superficial; it's only worth 1 point (out of 5). They turn in this attempt at the start of tutorial, and are given a blank copy of the problem to work on. The students then work on the problem in groups of 3-4 at the chalkboards. (This classroom is in the old math building so it has lovely chalkboards filling three walls.) Different groups then explain to the class their suggested answers to the different parts of the problem, and students discuss these answers. They also discuss how the problem might be adapted or modified for use in different settings, for example, changing the organism so it can be reused on a test, or making part of it into a shorter stand-alone problem. (In future we'll try to get them to also explicitly discuss what is needed for a good written answer to the problem, but they're not ready for that yet.)
All this seems to be OK with them. But the final step is for each student to write out a careful answer to the problem they've been discussing, as if this was an exam setting. These answers are handed in and marked; they're worth 4 points. At present the group work is left on the chalkboards while students are writing their answers, but I've told them that in a few weeks we'll start erasing the boards before they write their answers.
Students are complaining that this is a waste of their time, that they don't learn anything by having to write answers after they've already seen how the problem should be answered, and that they would learn more by spending the time in additional discussion. I disagree - I think that observing the right answer doesn't lead to much learning, and that having to apply what they've just observed by creating a written answer adds a lot.
In tomorrow's lecture I'm going to show them some data that might help them see the value in this. It's from a paper that just appeared in Science (Karpicke and Blunt). In both of the two studies they describe, the authors had students spend 5 minutes reading a half-page of text about a biological topic, and then consolidate what they'd read in various ways. The students were then asked to predict how much they would remember a week later. A week later they were tested on each topic.
In the first study the students either (i) did nothing more, (ii) reread the text three more times, (iii) spent 25 minutes making a concept map with the text, or (iv) tested their recall immediately by writing about it for 10 minutes, then reread the text, and retested their recall. In the second study the students either (v) spent 25 minutes making a concept map or (vi) tested their recall, reread the text, and retested their recall. In the first study each student read only one text and was tested a week later with a short-answer test. In the second each student was given two texts, one learned with a concept map and one with recall testing, and these were tested a week later using either a short-answer test or a concept map (in randomized combinations).
In both studies the students predicted that they'd remember more with the non-testing methods, but in the post-tests they always scored substantially higher when they had consolidated their reading by testing their recall. Here are edited versions of their graphs:
 |
| All the data |
 |
| Part of the data, that I'll describe to the students |
I'm going to show my students this study in tomorrow's lecture, and I'm going to give them two conclusions: First, people are not very good judges of how much they've learned. (So my students should realize that their opinions of how much they learn by different tutorial activities may well be mistaken.) Second, testing oneself is an excellent way to learn. (So my students should realize that having to develop a written answer after a discussion is a valuable way to reinforce what they've discussed.)
This will take a few minutes that I could otherwise spend talking about mitosis but I think learning how to learn is more important. The students have a mini-midterm coming up on Friday, so they should be fairly receptive to ideas about how to learn. I don't expect that this new data will convince them all that my tutorial design is good (that's why I wrote 'should' above instead of 'will') but at least they'll realize that I'm not just doing it to to be mean.
Saturday, January 22, 2011
Genetic mapping
In yesterday's course meeting for my new second-year genetics course (which I'm now thinking of as "21st Century Genetics"), I mentioned that the syllabus doesn't include the classical technique of genetic mapping. The others were shocked!
My students will learn how meiosis works. They'll learn about segregation and independent assortment. I've never really seen clear explanations of the meanings of these widely used terms, but segregation means that each daughter cell gets one version of the two homologous chromosomes (never two or none), and independent assortment means that which version of each pair a particular cell gets is random and independent of the version it got of each other pair. They'll learn how crossing-over between parts of a pair of homologous chromosomes makes new combinations of the alleles.
The students will learn how to find out if genes are linked (close enough together on the same chromosome that their alleles aren't randomized by meiotic assortment and crossing-over). They'll also learn that the frequency of crossing-over between any two genes gives a rough estimate of how far apart they are. They might even learn how to compare these frequencies to tell which gene is in the middle of a group of three linked genes (maybe as a homework problem). BUT, they won't learn to use three-factor crosses to determine 'map distances'. (Here's a web page with a fill-in-the-boxes version showing how such mapping analysis is done.)
Why not? Because they won't have any use for this skill. Even if 1000 students take the course each year, I would be very surprised if even one ever needed to map genes using crosses, except as an exercise in an old-fashioned upper-level genetics course.
Here's a page arguing that even real geneticists didn't do this - that the idealized three-factor mapping cross was largely an exercise for students. I don't think that's necessarily true, but it's certainly true that real geneticists rarely do this any more. Genetic mapping in general, and mapping by three-factor crosses in particular, is fast becoming an archaic technique. If one of my students should ever find that they need to do this (and I'm having a hard time coming up with an example where they would), there are lots of textbooks to show them how.
I think that the main reason genetics courses have always included three-factor mapping is that (i) this used to be how accurate gene maps were made, and (ii) this provides a tidy way to test whether students understand the consequences of crossing-over.
I think I will teach the students the difference between a physical map and a genetic map of a chromosome, and I'll expect them to be able to explain why the two kinds of map might not be identical - because recombination frequencies are influenced by DNA sequences (chromosomes have hotspots and cool spots), and because the data from the crosses may have flaws (low numbers, phenotypic problems that limit detection of recombinants). But I won't expect them to be able to do the mapping.
My students will learn how meiosis works. They'll learn about segregation and independent assortment. I've never really seen clear explanations of the meanings of these widely used terms, but segregation means that each daughter cell gets one version of the two homologous chromosomes (never two or none), and independent assortment means that which version of each pair a particular cell gets is random and independent of the version it got of each other pair. They'll learn how crossing-over between parts of a pair of homologous chromosomes makes new combinations of the alleles.
The students will learn how to find out if genes are linked (close enough together on the same chromosome that their alleles aren't randomized by meiotic assortment and crossing-over). They'll also learn that the frequency of crossing-over between any two genes gives a rough estimate of how far apart they are. They might even learn how to compare these frequencies to tell which gene is in the middle of a group of three linked genes (maybe as a homework problem). BUT, they won't learn to use three-factor crosses to determine 'map distances'. (Here's a web page with a fill-in-the-boxes version showing how such mapping analysis is done.)
Why not? Because they won't have any use for this skill. Even if 1000 students take the course each year, I would be very surprised if even one ever needed to map genes using crosses, except as an exercise in an old-fashioned upper-level genetics course.
Here's a page arguing that even real geneticists didn't do this - that the idealized three-factor mapping cross was largely an exercise for students. I don't think that's necessarily true, but it's certainly true that real geneticists rarely do this any more. Genetic mapping in general, and mapping by three-factor crosses in particular, is fast becoming an archaic technique. If one of my students should ever find that they need to do this (and I'm having a hard time coming up with an example where they would), there are lots of textbooks to show them how.
I think that the main reason genetics courses have always included three-factor mapping is that (i) this used to be how accurate gene maps were made, and (ii) this provides a tidy way to test whether students understand the consequences of crossing-over.
I think I will teach the students the difference between a physical map and a genetic map of a chromosome, and I'll expect them to be able to explain why the two kinds of map might not be identical - because recombination frequencies are influenced by DNA sequences (chromosomes have hotspots and cool spots), and because the data from the crosses may have flaws (low numbers, phenotypic problems that limit detection of recombinants). But I won't expect them to be able to do the mapping.
Saturday, January 15, 2011
Genetics problem for tutorial discussion
The postdoc and I just created an excellent genetics problem for the pilot section of my new course.
The problem needed to get students thinking about how changes to genes affect phenotype, but it couldn't involve crosses because they won't be doing those for another couple of weeks. That rules out just about all the problems in the textbooks.
This new problem has everything:
What do you think this fabulous problem is about?
The problem needed to get students thinking about how changes to genes affect phenotype, but it couldn't involve crosses because they won't be doing those for another couple of weeks. That rules out just about all the problems in the textbooks.
This new problem has everything:
- haploinsufficiency
- dominance
- repressor gene
- activator gene
- natural polymorphism
- important human diseases
- screening of newborns
- problems important in developing countries
- amino acid substitutions
- isoelectric focusing to detect changed protein charge
- mixed-allele dimers
- differences in protein levels
- developmental regulation
- interactions between fetus and mother at the placenta
- suppressor mutations (mitigating the deleterious effects of another mutation)
- natural selection in human populations
- mutations that are very well characterized (DNA, RNA, protein, function)
- genome-wide SNP analysis
- a mutation that's lethal when homozygous but beneficial when heterozygous
- new research in a high-profile journal (Sept. 2010 paper in Nature Genetics)
- students label subunits in tetramers
- students predict bands in gels, for different genotypes and developmental stages
- students predict protein levels through human development (draw lines on graph)
- students diagram regulatory interactions between genes, for different genotypes
What do you think this fabulous problem is about?
Saturday, January 08, 2011
Teaching about 'dominance'
In my new genetics course I'll soon be teaching about how genotypes determine (or influence) phenotypes in diploid organisms. For these Week 3 classes I want to give the students some reading material, both to read before the lectures and as a study reference for material covered in class. But there's nothing suitable in any of the genetics textbooks I've looked at, so I need to create it myself. Below I'm going to try to work out how best to present this and to design the reference I'll have them read.
The Week 2 lectures (= this week), will discuss natural genetic variation, how mutations generate this variation, and the phenotypic consequences of genetic differences in haploids and homozygous diploids. In the last of these lectures I want to consider the differences caused by standing genetic variation as well as lab examples. And here I should raise the issues we'll deal with next week, explaining that diploidy complicates the relationship between genotype and phenotype, and that the next week's classes will all focus on building a solid understanding of this relationship in diploid organisms.
Somewhere (in the Friday Week 2 class or in the Monday Week 3 class) we'll need to consider that there are different kinds of phenotypes. Some are strictly qualitative - presence or absence of an antigen or blood type, presence or absence of a disease - but many are best treated as quantitative, especially when we consider natural variation. These include obvious things like height and hair colour, and less obvious things like about of an enzyme or metabolite present in a cell or bodily fluid.
I'll also need to introduce the idea of 'risk' as a quantitative phenotype - this is best done in the context of natural variation and genomics.
The first Week 3 class will just be about interactions between alleles of single genes. I'll start with some of the same examples I used the Friday before, asking students to predict the phenotypes of individuals heterozygous for mutations whose homozygous phenotypes we've already established. These should include intermediate phenotypes, 'both-type' phenotypes, and dominant/recessive phenotypes, and genes with more than two alleles.
The existing terminology is terrible, since everything is described in terms of dominance, whereas dominance and recessiveness are really only two extremes of the range of heterozygous effects. The problem is maintained by the practice of beginning genetics courses with Mendel, and of introducing all the important concepts with dominant/recessive allele pairs and the A/a allele representation. Only long after students learn this (mainly by rote) are they told about genes with more than two alleles and about 'Variations on Dominance' (Introduction to Genetic Analysis), 'Modifications of Dominance Relationships' (iGenetics), or 'Complications in the Concept of Dominance (Genetics: Principles and Analysis). These books, and all the other genetics textbooks I've seen, present 'co-dominance' and 'incomplete dominance' or 'incomplete dominance'
Oh, and in the preceding Friday class I also need to raise the important issue of how we name alleles - when the A/a convention is appropriate and when it isn't. I'll tell them that its usually only appropriate for made-up examples in classrooms, because genetics researchers have different conventions for the real organisms they study. (There's no point teaching students these conventions, because they are not only arbitrary but are different for different organisms.) I'll also tell the students that I will only use the A/a convention for alleles known to be dominant/recessive to each other, and that they should be careful to only use them it they are confident that this is the case.
I really wish we had good terminology for the different kinds of effects. I don't want to use 'codominant' and 'semi-dominant' (or 'incompletely dominant'), but the only alternative is to describe the actual relationship in each case. Maybe I can at least standardize the words I'll use in this course: 'blended' for a heterozygote phenotype that's halfway between those of the homozygotes, 'both phenotypes' for co-dominance.
The Week 2 lectures (= this week), will discuss natural genetic variation, how mutations generate this variation, and the phenotypic consequences of genetic differences in haploids and homozygous diploids. In the last of these lectures I want to consider the differences caused by standing genetic variation as well as lab examples. And here I should raise the issues we'll deal with next week, explaining that diploidy complicates the relationship between genotype and phenotype, and that the next week's classes will all focus on building a solid understanding of this relationship in diploid organisms.
Somewhere (in the Friday Week 2 class or in the Monday Week 3 class) we'll need to consider that there are different kinds of phenotypes. Some are strictly qualitative - presence or absence of an antigen or blood type, presence or absence of a disease - but many are best treated as quantitative, especially when we consider natural variation. These include obvious things like height and hair colour, and less obvious things like about of an enzyme or metabolite present in a cell or bodily fluid.
I'll also need to introduce the idea of 'risk' as a quantitative phenotype - this is best done in the context of natural variation and genomics.
The first Week 3 class will just be about interactions between alleles of single genes. I'll start with some of the same examples I used the Friday before, asking students to predict the phenotypes of individuals heterozygous for mutations whose homozygous phenotypes we've already established. These should include intermediate phenotypes, 'both-type' phenotypes, and dominant/recessive phenotypes, and genes with more than two alleles.
The existing terminology is terrible, since everything is described in terms of dominance, whereas dominance and recessiveness are really only two extremes of the range of heterozygous effects. The problem is maintained by the practice of beginning genetics courses with Mendel, and of introducing all the important concepts with dominant/recessive allele pairs and the A/a allele representation. Only long after students learn this (mainly by rote) are they told about genes with more than two alleles and about 'Variations on Dominance' (Introduction to Genetic Analysis), 'Modifications of Dominance Relationships' (iGenetics), or 'Complications in the Concept of Dominance (Genetics: Principles and Analysis). These books, and all the other genetics textbooks I've seen, present 'co-dominance' and 'incomplete dominance' or 'incomplete dominance'
Oh, and in the preceding Friday class I also need to raise the important issue of how we name alleles - when the A/a convention is appropriate and when it isn't. I'll tell them that its usually only appropriate for made-up examples in classrooms, because genetics researchers have different conventions for the real organisms they study. (There's no point teaching students these conventions, because they are not only arbitrary but are different for different organisms.) I'll also tell the students that I will only use the A/a convention for alleles known to be dominant/recessive to each other, and that they should be careful to only use them it they are confident that this is the case.
I really wish we had good terminology for the different kinds of effects. I don't want to use 'codominant' and 'semi-dominant' (or 'incompletely dominant'), but the only alternative is to describe the actual relationship in each case. Maybe I can at least standardize the words I'll use in this course: 'blended' for a heterozygote phenotype that's halfway between those of the homozygotes, 'both phenotypes' for co-dominance.
Saturday, November 13, 2010
Do as I say, not as I do?
Very few of the speakers at the Reinvention Center conference have applied any insights into teaching and learning into their presentations. We get the same talking heads, and today a bad powerpoint.
Global health
Note: This session turned out to be a presentation about how and why to develop courses and programs in global health, not about how to bring global health issues into ones own courses, as I was expecting. She didn't give us any outline or goal for the talk at the beginning, so it took me a while to figure this out.
Aha - she (Heather Wipfli, USC) starts by asking us to write definitions.
Traditionally:
Aha - she (Heather Wipfli, USC) starts by asking us to write definitions.
- Small audience but good ideas.
Traditionally:
- rural
- babies and mothers
- infectious diseases
- vaccines
Themes:
- We (the rich) help others (the poor)
- Help needed by helpless innocents, not prostitutes
- fix and move on
Who does it:
- community saviours/superstars
- real (Hollywood) celebrities
Changing the perspective:
- We are all getting older.
- Chronic diseases (cancer, cardiovascular, asthma, diabetes) kill most people even in low-income countries.
- Epidemiology is changing
- Global warming and environmental issues
- Urbanization increasing but megacities not like NY and LA.
So how should we change our academic programs?
Need to shift student perspectives from medicine to global health. Many universities have graduate programs in global health (mostly MSc and MPH), but the number of associated undergraduate programs is increasing rapidly.
Geoffrey Rose: "Sick individuals arise from sick populations."
Every factor is interrelated with many other factors and various levels, from domestic to international.
Mexico (70% overweight, 28% obese) consumes 665 servings of Coke per capita per year.
Students are highly motivated by these issues. How can we integrate this into our courses?
Change of topic: using electronic games to teach. The Redistricting Game (teaches about political boundaries).
For genetics: discuss (have them suggest/investigate) gene-environment interactions that apply to immigrant populations.
The Creative Campus - catalyzing non-routine engagement
Still at the Reinvention Center conference, Steven Tepper and Elizabeth Long Lingo, speaking after lunch.
First speaker:
Stupid jokes by a speaker (topic=creativity) who admits that he has too many slides. Now he's reading his slide to us. Now a slide with a table full of 4-digit numbers... And another... Dorky animations, annoying music.
Uptalking!
A Master of Fine Arts degree may give transferable skills for the aestheticization of everyday life (IKEA, Apple, Target toaster).
Second speaker:
Curb Program in Creative Enterprise and Public Leadership:
Undergraduate Scholars program (4 years):
First speaker:
Stupid jokes by a speaker (topic=creativity) who admits that he has too many slides. Now he's reading his slide to us. Now a slide with a table full of 4-digit numbers... And another... Dorky animations, annoying music.
Uptalking!
A Master of Fine Arts degree may give transferable skills for the aestheticization of everyday life (IKEA, Apple, Target toaster).
Second speaker:
Curb Program in Creative Enterprise and Public Leadership:
Undergraduate Scholars program (4 years):
- ability to invent
- expressive agility, to convince
- dexterity to develop and implement their ideas
- ability to critically examine.
Subscribe to:
Posts (Atom)