Pilot Study Report about Data Collection Instruments

Published:

The aim of the pilot study was to examine data collection instruments that will be used in the main study and main data collection. It is important that any tools of data collection should have a pilot study 'in order to iron out any problems of overlap of categories' ( Cohen et al, 2005: 130). In other words, Drever (1995: 56) points out that the researcher in the pilot study is trying to do two things: 'first, to give the interview a trial run under realistic conditions; second, to get as much information as possible from the other person about how they interpret and reacted to the researcher's questions.

Instruments

Two instruments of data collection were used in the pilot study: first, in class-essay writing sessions, this was done by asking students to write three essay drafts in three different writing sessions. The time allowed for writing the essay drafts was 90 minutes. The number of errors was used to measure the achievements of the students' accuracy in a new piece of writing. Second, conducting an interview with students first and then their tutors in order to know their expectations towards tutors' error feedback.

Participants

Lady using a tablet
Lady using a tablet

Professional

Essay Writers

Lady Using Tablet

Get your grade
or your money back

using our Essay Writing Service!

Essay Writing Service

Students from the two colleges were selected and had informed that their writing work would be analysed and used for research and teaching purposes (See table 1 and 2). First year students and second year students from each college were enrolled in three writing class sessions. The total number of students who participated in the study was ninety-seven students. The students assigned to two different groups: first group, students who received 'coded' direct written feedback focused on form and the other group who received 'un-coded' written feedback focused on form as well. English writing tutors were selected in the study and their number was four (See table 3).

College One Groups

First year

Second year

Direct Written Corrective Feedback

11

14

Indirect Written Corrective Feedback

12

14

Table: 1

College Two Groups

First year

Second year

Direct Written Corrective Feedback

11

12

Indirect Written Corrective Feedback

11

12

Table: 2

Tutors NO

First year

Second year

College One

1

1

College Two

1

1

Table: 3

Pilot Study Procedures and methods of collecting data Used

The researcher talked to the head of the teachers of the two colleges and had their permissions to enter the premises. The students were told about the pilot study and the process, of how they would receive their corrective essay drafts. After tutors had explained the process of the pilot study, they asked them to participate without pushing them forcefully to join the two groups. Tutors and the students had been asked to sign the consent forms. Three second year students refused to sign the consent forms and participated in the study in college one. Their refusal was based on obtaining bonus marks participating in the study; therefore, those students were excluded from the study. Some of the students of the two groups in college one and two asked for bonus marks as well, but then they accepted to participate when their tutors explained how this study would be beneficial for Libyan education system and how it would improve the quality of the university pedagogical decision making in Libya. Students were required to write three descriptive essays on different topics but on a similar context during 6 weeks' time (See table 6). Essay topics were on different topics because students would be familiar with the text and the chance of having accuracy improvement in the next draft would be high (See table 4). Tutors were asked to focus on correcting errors based on form, and they were given a list of types of errors that should be addressed on students' essay drafts. (See table 5).

Essay Topic

No

First Year

Second Year

1

Describe your family. Is it a typical family?

Describe any important Muslim event in your country. Why is it important?

2

Describe your best friend. Why is he/she the best of your friends?

Lady using a tablet
Lady using a tablet

Comprehensive

Writing Services

Lady Using Tablet

Plagiarism-free
Always on Time

Marked to Standard

Order Now

Describe your national holiday event in your country. Why is it important?

3

Describe one of your parents. How would you describe her or him?

Describe your favourite holiday place in your country. Why is it favourite one?

Table: 4

Article Errors

Plural Nouns Errors

Prepositions

Wrong Word

Punctuations

Missing word

Spelling

Word Order

Capitalization

Subject-Verb Agreement

Personal Pronouns

Verb Tense Errors

Incomplete Sentence

Table: 5

Tutors asked students to write between 250 - 300 words for the first essay draft for both first and second year students. First year students were given 90 minutes to describe their families, and whether they are typical families in Libya. Similarly, second year students were given 90 minutes to describe any important Muslim event in their country and why it is important to them. Each student essay was then handed in to the tutor. Tutors were given time to correct essay drafts based on the direct and indirect written corrective feedback techniques. On each of students' essay drafts errors were coded with the correct form of the error for the direct written feedback group. The indirect written feedback group had their errors underlined in red without giving the correct form of the error. After the essay drafts were corrected, each essay draft was photocopied before it was handed back to the students to revise. Students were given their original essay drafts in the next writing session and were asked to revise their corrected errors. A week later and after students had been given time to revise their essay drafts, first year students were asked to write another descriptive essay draft on their best friends and why they were the best of their friends. Second year students were asked to describe their national holiday event in their country and why it is important. First and second year students were asked to write between 250 - 300 words for the second essay draft and were given 90 minutes to write their essays. Each student essay draft was then handed in to the tutor after the completion of the estimated time given to the students. Tutors corrected the second essay drafts based on the direct and indirect written corrective feedback techniques. Photocopies of the second essay drafts of the students were taken for analysis. In the following session students were given their corrected second essay drafts and asked to revise their underlined error forms and corrected error forms. Tutors continued providing students with two types of the written corrective feedback looking at their accuracy improving of subsequent writing. Every time students submitted their essay drafts tutors corrected and handed the essay drafts back to students to revise. In week 6, students were asked to write the third draft and the last one. First year students were asked to describe one of their parents. How they would describe them while second year students were asked to describe their favourite holiday place in their country and why it is favourite one. All students were asked to write between 250 - 300 words for the final essay draft and were given 90 minutes to write their essay drafts. Tutors collected the essay drafts and correct these essays mark them and return them back to the students the following writing session. Photocopies were taken from the final essay drafts of the students for analysis. Improvement on students' accuracy in the second and the third students' essay drafts will be analysed.

Academic Year

Essay One

Essay Two

Essay Three

First Year

Beginning of week 1

2nd session a week later

End of Week 6

Second Year

Beginning of week 1

2nd session a week later

End of Week 6

Table: 6

Concerns and suggestions raised during data collection of the pilot study.

Three students refused to participate in the study, unless they had given bonus marks. These students were excluded from the study.

The other students who asked for bonus marks to participate in the study were accepted to participate in the study without getting any bonus marks when their tutors explained the importance of the study to the education system in Libya.

Lady using a tablet
Lady using a tablet

This Essay is

a Student's Work

Lady Using Tablet

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

Examples of our work

It had been noticed that few students who could write up to 300 words while the others were unable to reach 250 words. The number of words required for the essay writing should be reduced to 150 -200 words.

Two students were unable to finish their essay within the 90 minutes and asked to give them extra time. I negotiated the issue with their tutor, and we agreed that the 90 minutes given is adequate and fair enough for low level students to write their essay in-class sessions. Those two students had written more than 150 words in total and thus 90 minutes are enough.

Few students show no interest writing their essay insufficiently as they wrote few lines. After their tutors had assessed these essay drafts, and I was told that they did not do their best. I suggest that the final essay draft should be assessed based on the exam so the students would show their effort towards the quality of their writing.

It had been noticed that one of the tutors did not show or recognise his students' errors so tutors' experiences CV is needed for the main study.

Interview with students and then with their tutors

The second step used was to conduct the interview with students and tutors. The aim of the interview was to document the students and tutors' expectations concerning tutors' written feedback on students' essay drafts. The place of the interview was selected in consultation with the heads of the English departments and was prepared to carry out the interview individually with students and tutors. They were very happy to assist me; therefore, they offered me a small quiet office, free from interruptions to use it for the interview with the participants. It was obvious that before planning the interview students and tutors should not have any circumstances that may hinder them to remain in the college and that would lead to the absence of the participants. The plan was to interview two participants each day and that was because of the availability of the students and the tutors during their attendance in the college. The other reason was that interviewing two participants each day would take approximate two hours. This was enough to conduct the two participants' interviews without feeling exhausted or reluctant. I noted down the timetable of the students' courses first so I would be able to interview them when there were not classes running. Students' selection was randomly, one student from each class was chosen, and that would be two students from each college (See table 7). However, I informed the participants with the dates and time available during the week. I gave them the choice to choose the day and time they like. Tutors' timetable was taken as well, and they were informed with the dates and time available. Participants were all informed that the time of each interview would normally take from 30-40 minutes, and they were also informed that their interviews would be audio-recorded. Their permission to use the digital audio-recorder during the interview was taken. Later, I had received their acceptance with the dates and time they were available to them. The second day was the first interview with the first two participants. I came early one hour before the first interview started. Things like air conditioner and office lightening were necessarily checked before conducting the interview. Digital audio-recorder was verified that it was working in a good condition, and the pens and a notebook was on the table. Food and drink were brought to help the participant to relax.

Interview Groups

First year

Second year

Direct Written Corrective Feedback

1

1

Indirect Written Corrective Feedback

1

1

Table: 7

I opened the door of the small office, and I left it open so the participants would know that I was available and there in the office. As soon as the time of the interview had started, I was almost ready for the interview. The first participant came in, and I asked him to have a seat. A semi-structured interview was chosen as an appropriate method; it enables students and tutors to discuss and debate their experiences and views in more details. I started the interview to explain in more details, what was required from the interview, and how the participants' role is important by revealing their concerns and sharing their views? Then I followed my start with guided questions that were planned. In order to give the interview questions positive and competent achievements, the interview questions were first written in English and then were translated into Arabic, the native language of the interviewees. Professionals translated the Arabic translation again into English in order to check the accuracy of the translation. The interview questions had revised again and the translation had been confirmed before conducting the interviews with the students. The interview was not controlled by these prepared questions rather than to help the interviewer to probe deeply depending on the answers of the participants. The interviewer was able to interact with the interviewees and freely added additional questions. Notes had been taken, were separated, and followed the order of the questions. This method was adopted to help the interviewer to sort out what had been noted from the interview conversation. I almost asked every planned question to the interviewees and new questions had been generated during the interview conversation. This method was used with all participants and the interview length had been varied (See tables: 8, 9, 10, and 11).

The interview with students carried out within four days and the interview atmosphere was encouraging. The interviewees were enthusiastic and their attitudes towards the interview were positive. However, Tutors were informed and had given the time and venue of the interview. They all had agreed with the time given, and they were informed with the approximate length of the interview between 30-40 minutes. Tutors' interviews were carried out within two days and those were two participants each day. No negative side of their attitudes was observed, and they were all cooperative with the interviewer. They were informed with the concerns had been given from the students' interview. New questions had been generated as the interviewees shared their experiences and knowledge towards tutors' written feedback.

College One

First Year Students

Interview duration

Student one with Direct CF

35 minutes

Student two with Indirect CF

44 minutes

Table: 8

College One

First Year Students

Interview duration

Student one with Direct CF

45 minutes

Student two with Indirect CF

40 minutes

Table: 9

College Two

Second Year Students

Interview duration

Student one with Direct CF

45 minutes

Student two with Indirect CF

37 minutes

Table: 10

College Two

Second Year Students

Interview duration

Student one with Direct CF

35 minutes

Student two with Indirect CF

45 minutes

Table: 11

College One

Tutors

Interview duration

First Year Tutor

50 minutes

Second year Tutor

57 minutes

Table: 12

College Two

Tutors

Interview duration

First Year Tutor

66 minutes

Second year Tutor

52 minutes

Table: 13

Concerns and suggestions raised during the interview with students and tutors.

Students who participated in the interview were willing to share their views and concerns about their experiences.

It had been noticed that time assigned to the interview was shorter than it was expected; therefore, time should be extended from 30-40 minutes into 40-60 minutes.

Students' questions were understandable and students found it easy to answer with the help of the translation. The translation was used when it is necessary. Therefore, interview questions were appropriate for the interview.

There were two concerns with the tutors' interview questions. Question number one was changed from 'what types of written feedback do you think are helpful for students to improve their essay writing skills? Why?' in 'What type of written feedback do you think are helpful for students to improve their essay writing skills? Why?'

Tutors answered Question number 6 'Have you observed any midterm effects on students' essay writing as a result of using different types of written feedback? What are those effects, if any?' depended on their experiences not on what they had observed during midterm. This was because they could give their opinions if it was a midterm-based exam.