The early eighties with the publication of the cockroft

Published:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

In the early eighties with the publication of the Cockroft report, recommendations were being made for greater emphasis on discussion within the mathematics classroom. The positive benefits of both pupil teacher discussion and pupil pupil discussion within the mathematics classroom were seen as a useful tool in creating more solid understanding of concepts and ideas. (Cockroft, 1982).

With the introduction of the National Curriculum and the Framework for Teaching Mathematics, the development of mathematical reasoning, explanation and language skills was encouraged, the practical implications that this elicits with respect to time factors, can create difficulty within the classroom. As Practitioners in education, with the current pupil centred policy, we are duty bound to allow all of our students a platform to offer their explanation, as well as the mechanism to enable them to carry out worthwhile mathematical discussions. This study sets out to explore the potential that recent developments, in Artificial Intelligence and Natural language processing, may have in providing a platform that enables students to develop their metacognitive processes through interaction with Chatbot technology.

Introduction

We live in an increasingly technology reliant age, an age where computers are present in virtually every aspect, within the home, at work, in the cars we drive and taking a greater role in our leisure time, but in the majority of these situations we are still merely utilising the computer in a servile role. It can be claimed that this was the role for which it was originally created, but with the exponential growth in its computational capabilities, a use in which it is quickly outgrowing.

Charles crook (crook, 1987) examined and assessed four frameworks for computer use within schools; computer-as-resource, computer-as-tutor, computer-as-pupil and the computer-as-fabric. Crook concluded that of the four frameworks he examined he believed that the first three could not transform the structure of typical school life, as the computer could not offer the contextual support needed within learning. The fourth framework that he examined, computer-as-fabric, was felt to be the model that could revolutionise education. Crook believed that the use of the virtual classroom held the greatest potential benefit to learning, by creating an environment whereby classroom type activities could take place in determinant of the location of the student.

More recent research has re-enforced crooks findings, leading researchers to further conclude that the use of computers within the mathematics classroom offers essential benefits to pupils, with regard to the conceptualizing of abstract concepts and ideas.

Research undertaken in recent years has shown that the use of Computer Mediated Communication(CMC), within an educational context, creates a greater feeling of empowerment within pupils. This greater level of confidence shown by the pupils themselves also can be demonstrated by the greater levels of participation with CMC shown by pupils in comparison to the more traditional Face to Face discussion methods employed within the classroom. (Khan, 2006).

Even though research points to such a positive benefit to the usage of Virtual Learning Environments (VLE's) and associated technologies within Secondary Schools, the number of secondary schools fully embracing this resource is minimal. In the IMPACT2 study carried out by BECTA in 2001/2, Keystage 3 Mathematics had the lowest utilization of ICT resources in its delivery; this was also reflected in the pupils Home usage of Internet and computer resources. (BECTA, 2002)

Linguistically, there is a lot of discussion with respect to the status of communication via CMC. The language used, although in written form, contains few of the formal rules normally contained within written language. Baron, in a study of instant messaging amongst American students, concludes that from a linguistic point of view, instant messaging and on-line chat has created a new form of language discourse that is a hybrid of both written and spoken language, whereby it uses the written form but employs many of the rules normally associated with spoken language. (Baron, 2005).

It is my belief that, with the expansion of the capabilities of computers since crooks research was carried out, that of the four frameworks there are now three frameworks that have the potential to revolutionise education. As well as the concept of the computer-as-fabric I believe that we should also consider two more of the frameworks.

These are:

Computer-as-tutor: With the growth of computing technology over the past 20 years we have seen an increased availability of applications that can act in an interactive way with the individual pupil, aiding the pupils learning and progression. One example is the success maker software suite. Success maker is normally used to boost pupils learning within Maths and English, demonstrating and working through questions with pupils before allowing pupils to build on this knowledge. The software is capable of 'remembering' a pupil's progress, their strengths as well as their weaknesses and each time the pupil logs on to use the software these criteria are used to aid the pupils' progression.

Computer-as-pupil: The computer-as-pupil framework is the focus of this study and examines the use of Chatbots to enable conversational dialogue between pupils and the computer. Vygotskys theory on zone of proximal development states that the very act of interaction promotes a pupils ability to develop their own understanding of concepts, can we as practitioners utilize the recent developments within Artificial Intelligence to mimic conversational interaction? Alice Kerly examined the use of Chatbots within open learner models of Intelligent tutoring systems and concluded that this type of technology had the capabilities of providing useful dialogue between the user and the computer.(kerly,2006).With the rise in a computers power, in programming techniques and capabilities can the computer-as-pupil become a useful tool?

Just as the potential educational uses of the computer have changed dramatically over the intervening years since Crooks initial research in 1987. We are now at the stage in the computers evolution that an increase in the number of new innovative methods for there use within education is about to take place.

This study sets out to examine the potential use of Artificially Intelligent Chatbot technology as an educational tool, examining its potential to aid the development of metacognitive abilities within pupils.

Theoretical Context

The great importance that must be placed upon conversation and discussion within an educational context has a sound basis within social-constructivist theories of learning.

The very act of conversation allowing the student to gain a greater awareness of their own thought processes. This awareness within the context of the concept being discussed is termed as 'metacognition'. The student, through discussion is no longer just learning but becomes aware of their cognitive processes that are taking place during that learning.

Vygotsky through his theory on the zone of proximal development, discussed the idea that at all times during learning a student posses a level of actual achievement, that is things that can be achieved at the present time, without any further help, and a potential level of achievement which is what can be achieved through interaction with another, be that the teacher or a more experienced peer.

Vygotsky saw the progression of learning as a cycle which initially begins through social interaction and as the learning becomes more concrete within an individual moves toward the more internalized self-regulated cognitive processes that we can term as metacognition. (Vygotsky.1978). The learner moves from open discussion toward talking themselves through the learning out loud, and then onto internalizing this speech at which point this then becomes the learners 'actual level of achievement'.

"Metacognition refers to one's knowledge concerning one's own cognitive processes and products or anything related to them, e.g., the learning-relevant properties of information or data. For example, I am engaging in metacognition (metamemory, metalearning, metaattention, metalanguage, or whatever) if I notice that I am having more trouble learning A than B; if it is, it strikes me that I should double-check C before accepting it as a fact; if it occurs to me that I had better scrutinise each and every alternative in any multiple-choice type task situation before deciding which is the best one; if I sense that I had better make a note of D because I may forget it;.... Metacognition refers, among other things, to the active monitoring and consequent regulation and orchestration of these processes in relation to the cognitive objects or data on which they bear, usually in the service of some concrete goal or objective" (Flavell, 1976, ).

Metalanguage

The term metalanguage refers to the language used by students to describe the cognitive processes that they undertake, and although they may contain words or phrases that are found within a general language, they may, for the purposes of describing the concept within that particular discussion, convey different meanings.

Mathematics through its very nature leads us to use metalanguage when describing cognitive reasoning. A student may use the term 'share' when thinking through a division problem and this may be verbalised and then corrected when discussing the problem with peers or the teacher. In this example the term 'share' is a piece of metalanguage. Although the use of metalanguage is not the primary focus of this study it will be interesting to examine the language that has been used by the students to 'teach' the Chatbots and whether any occurrences of metalanguage appear within the Chatbot mind files.

Chatbot technology

Chatbot is a concatenation of the words 'Chat' and 'Robot', and is used within the computing community to describe a program that is able to converse with a user by utilizing Artificial Intelligence (AI) algorithms. Chatbots are another form of CMC technology that allow the user to interact conversationally with the computer as if they were conversing with another user

One of the first AI Chatbots to be developed was the 'ELIZA' Chatbot, which was programmed to mimic the types of dialogue that occur between a psychiatric counsellor and a patient. The 'Eliza' program utilized the Psychiatric technique of 'reflective questioning' which gave the impression of the user being in conversation with a 'real' person rather than a piece of computer software, The AI algorithms within the program were designed to parse the users dialogue, and create responses in the form of questions based on the users inputted conversation.

Eliza, although possessing a form of AI did not have the potential to learn from the conversations that it had undertaken with the user, several variants of Eliza appeared over the intervening years, including a variant which turned the original ELIZA concept on its head and responded as if it was an extremely paranoid person, reacting to the conversation initially in a normal way but as soon as the program interpreted any negativity within the users responses, became increasingly agitated in its responses to the users comments. Still the potential for the Chatbot to learn from its conversations remained elusive.

The development of the common programming language termed Artificial Intelligence Mark-up Language (AIML) spurred the programming community and the release of ALICE a Chatbot program based on the AIML language. This allowed the Chatbot programmer to introduce the ability for the Chatbot to do more than just de construct the users input and base a reply on its reconstruction; it gave them an ability to program a form of contextualisation into conversations.

If we examine the AIML code that drives an example chat we can start to see some of the limitations that this form of code holds.

< and > are used to contain programming tags

</ is used to close a statement set

The words in capitals are expected input or reply frameworks

<category><pattern></pattern><template></category> signifies an expected pattern of input and its response(s)

e.g. <category><pattern>CHEMISTRY</pattern><template> would be flagged by the program whenever a user inputted a sentence that used the word chemistry as long as it doesn't fit any other template contained within the program.

<srai><star/></srai> signifies a similar users response framework

Code example

<category><pattern>CHEMISTRY</pattern><template> <srai>MY FAVORITE SUBJECT IS CHEMISTRY <star/> </srai> </template></category>

<category><pattern>DO YOU KNOW CHEMISTRY</pattern><template> <srai>WHAT IS CHEMISTRY</srai> </template></category>

<category><pattern>DO YOU UNDERSTAND THERMODYNAMICS</pattern><template> <srai>WHAT IS THERMODYNAMICS</srai>

<category>

<pattern>WHAT IS CHEMISTRY</pattern>

<template>

The science of mixing chemicals.

<think><set name="it">

<set name="topic">CHEMISTRY</set>

</set>

</think>

</template>

Chat Example

User :what is chemistry

Chatbot: The science of mixing chemicals.

{direct response to user using the word Chemistry}

user :Tell me about chemistry

Chatbot: I can't find any info about that.

{user response not known by Chatbot}

User : chemistry

Chatbot: What makes it your favourite?

{Response to chemistry and' my favourite subject is chemistry '}

User :do you like chemistry

Chatbot:Sure, it is an interesting topic. Do you know Avogadro's Number?

{ Chatbot attempting to impart knowledge }

User :do you know chemistry

Chatbot:The science of mixing chemicals.

{ response to the question }

It can be seen from this example that the AIML group of programming languages have limitations in there ability to carry out a meaningful conversation with the user on one particular subject, without excessive writing of code by the programmer. In this example the simple response to the users input of 'chemistry' is controlled by approximately a dozen pieces of programming code, but as can be seen just through the use of everyday phrases that are unknown to the Chatbot creates unusual responses. In order for the programmer to cover all of the possible 'frameworks ' a question could be posed in, the use of wildcard 'flags' could be utilised, but even using wildcards the code needed would be extremely large and programmatically slow.

If we were then to introduce more complex dialogues using AIML coding techniques the code would become exponentially large and become impossible to implement.

The AIML language has been used to create some Chatbots with interesting 'personality' traits for example one which uses the lyrics of John Lennon songs to form the basis of its responses to user input.

There are available, commercial Chatbots such as Lingubot that use a framework of responses and concepts which are set by the purchaser, which determine the responses to a user's conversation. Although this could be classed as a form of learning on behalf of the lingubot, it is unable to learn from its subsequent conversations. To this end Lingubots and bots using similar technology tend to be termed as 'helper bots' or 'helper agents'. The lingubot technology is utilized by many companies within their on-line customer services, and progress is being made on developing Chatbots based on lingubot type technology that are able to interpret speech and can therefore be utilized within telephone services.

In order for this study to take a place a system was needed that did not utilize the AIML language system and had the capability to learn from its interactions with the user. A Chatbot was found that utilised a form of Natural language Processing to learn from its 'user Chatbot' interactions.

Billy Chatbot and Natural Language Processing

Natural language processing (NLP) is technique that allows a program to peruse a sentence, its structure and content in order to both extract meaning and create a response. The Chatbot selected to be used during this study was the 'Billy' Chatbot programmed by Greg Leedberg. Billy uses a basic form of NLP along with some AI algorithms developed by Leedberg to process conversations between the Chatbot and the user enabling Billy to 'learn' as a conversation progresses.

Billy Chatbot Features

Ability to save transcripts of chats

Ability to learn through interaction with user

Ability to return 'mind' to default

Ability to interchange the 'mind' files from one Billy Chatbot to another.

Ability to teach Chatbot through plain text

Default 'mind' file has basic arithmetical understanding

How does the program learn?

The program through the NLP creates associations between subjects and the words that are used around it, and is able, through interaction with the user, to learn from conversations that are held. This conversationally gained knowledge along with a programmed understanding of basic sentence structures, allows the program to divide learnt words into their various linguistic categories.

Example

The statement 'A Triangle has three sides' would firstly create associations between the words TRIANGLE, THREE and SIDES and then secondly create a subject word that is then associated with these words. So after learning that 'a triangle has three sides' we could ask the Chatbot the following:

USER: How many sides on a triangle?

To which the Chatbot will reply, because of the word associations created through the programs use of natural language parsing.

Chatbot: a triangle has three sides

Each time the subject word, in the case of the example 'Triangle', is used in close proximity to its associated words the association becomes stronger, this is carried out through the use of a 'counter flag', a numerical count of the number of times the association takes place. The use of a counter flag aids the AI algorithms to make decisions about the most likely response to a question or comment.

Methodology

The primary focus of the study was to examine the potential of using a Chatbot as a mechanism for aiding pupils' metacognitive abilities. In specific the students ability to accurately self assess their own understanding of the mathematical concepts covered during the time that the study took place.

As a piece of qualitative research a small group of pupils were selected randomly from the volunteers within a year 7 group. Another group of pupils were selected at the same time from the remaining volunteers to act as a control group. The control group were not made aware at the time of the study that they were taking part in the study, although once the study had been completed they were informed and consent was gained from the control group.

The study was set to take place in two parts;

The first part was to take place over three weeks and entailed the Study group attempting to converse with the Chatbot software, with the ultimate goal; within the mind of the study group; being that they would teach the Chatbot the mathematics that they had been learning over the three weeks that the study took place.

The second part of the study, involved firstly allowing both the study and control group to assess their own learning of the core 7 learning objectives and then both groups sitting the year 7 optional tests.

The control group and the study group were present in the same classroom, undertook the same lessons and were allowed the same levels of topic discussion as each other. The only difference being that the study group had access to the Chatbot to use in their own time.

Once the study was complete both groups were given the form on which to assess their own understanding of all of the core year 7 topics covered over the whole year, prior to sitting their optional year 7 tests.

Neither the study group nor the control group were informed that the study was examining metacognition; this was to deter the validity of the results being marred by the study group purposefully thinking about their cognitive processes.

Setting the study:

The study was carried out amongst a group of year 7 students who had access to computers at home; a group of five students were selected randomly from volunteers who were willing to take part in the study.

Each of the study group was given copies of the software and a brief tutorial on how to install and use the software. The group were informed that they were testing a piece of Chatbot software that would allow them to chat to their computer.

The study group were then given the following instructions about what was expected of them in order to test the software.

Chat freely with your Chatbot software, get to know the Chatbot

Try to teach the Chatbot as much of the maths that is covered in your lessons as you can. Explain the maths that you are learning in class in short sentences. (Example 'a square has four equal length sides.' Followed by ' a square has four right angles' )

chat to the Chatbot about maths

if the Chatbot says something that doesn't make sense or that you know is wrong say ' don't say that'

Ask your Chatbot questions about maths. Remember questions end with '?'

They were reminded that this was their own personal Chatbot and that it did not connect to the internet but allowed them to talk to the computer as if it was a person.

To further the illusion that the Chatbot was a way of communicating with the computer as if it was a person, the Chatbot was given a common identity.

The identity given to the Chatbot was:

Name : Freddy

Age: 12

Girlfriend : Becky

Town/Location : Brighton

The study group were left to 'test' the software in their own time without any input from the researcher. As much as was possible no mention was made of the software during the study period apart from direct questions from the members of the study group about problems that may have arisen with their copy of the software.

Once the three weeks that the study took place in were over the study group was reconvened, each member was given a short questionnaire to complete which covered how they felt the testing of the software went. This further emphasized that the study was purely concerned with the testing of a piece of software rather than a study into the pupils' metacognition. The questionnaire played no further part in the research that was undertaken

The group were also asked to email the researcher their Chatbots brain file and were given a set of instructions and a demonstration on how to accomplish this.

Analysis of results

Analysis of metacognition: Core objectives

In determining whether the Chatbot had affected a pupils metacognitive ability a comparison was made between the self assessed understanding of a core objective and the resulting pupils mark for the year 7 optional test in which that topic was covered. In doing so a comparison could be made between a pupils 'actual level of achievement' from the metacognitive perspective and from the data received through testing of the pupils 'level of achievement' through the year 7 optional testing. Differences between the control group and the study group could therefore be attributed to an outside influence which in the case of this study would be the effect the Chatbot had on the pupils' metacognition.

The Self assessment form, to aid simplicity for the pupils, involved the use of 'smiley's' in order to classify understanding, upon analysis the smiley's were quantified by applying a mark of 3 to smiley face, 2 to a neutral face and 1 to a sad face.

Table 1: study group self assessed metacognition of core objectives

Core Objective (study Group)

Student1

Student2

Student3

Student4

Student5

Simplify Fractions; identify equivalent fractions

3

3

3

3

3

Recognise equivalent fractions, percentages and decimals

3

3

1

3

2

Extend mental methods of calculation to include decimals, fractions and percentages

3

2

3

3

3

Multiply and divide 3 digit by 2 digit numbers. Multiply and divide decimals with 1 or 2 places by single digit whole numbers

2

2

3

2

3

Break a calculation into simple steps, choosing and using appropriate methods to solve the calculation

2

3

3

1

2

Check a result by considering whether it is of the right magnitude

1

2

2

3

1

Use letter symbols to represent unknown numbers

3

1

1

3

3

Know and use the order of operations, understand that algebraic operations follow the same order

2

1

1

2

2

Plot simple linear functions

3

3

1

3

1

Identify parallel and perpendicular lines, know the sum of angles on a straight line and in a triangle

3

3

3

3

3

Convert one metric unit to another, read and interpret scales

3

1

2

3

3

Compare two simple distributions using the range and one of the mean, median or mode

2

3

2

2

2

Understand the probability scale, find and justify probabilities based on equally likely outcomes in simple contexts

3

1

1

2

3

Solve word problems and investigate in a range of contexts, explaining and justifying methods and conclusions.

3

2

2

2

2

Table 2: Control group self assessed metacognition of core objectives

Core Objective (Control Group)

Student1

Student2

Student3

Student4

Student5

Simplify Fractions; identify equivalent fractions

3

3

2

3

3

Recognise equivalent fractions, percentages and decimals

3

3

2

3

3

Extend mental methods of calculation to include decimals, fractions and percentages

3

2

2

3

2

Multiply and divide 3 digit by 2 digit numbers. Multiply and divide decimals with 1 or 2 places by single digit whole numbers

3

3

2

2

3

Break a calculation into simple steps, choosing and using appropriate methods to solve the calculation

3

2

2

3

3

Check a result by considering whether it is of the right magnitude

3

3

2

2

2

Use letter symbols to represent unknown numbers

3

3

3

2

2

Know and use the order of operations, understand that algebraic operations follow the same order

3

2

2

3

2

Plot simple linear functions

3

2

2

3

2

Identify parallel and perpendicular lines, know the sum of angles on a straight line and in a triangle

3

3

2

3

3

Convert one metric unit to another, read and interpret scales

3

3

2

2

1

Compare two simple distributions using the range and one of the mean, median or mode

3

2

3

2

3

Understand the probability scale, find and justify probabilities based on equally likely outcomes in simple contexts

3

3

1

3

3

Solve word problems and investigate in a range of contexts, explaining and justifying methods and conclusions.

3

1

2

2

2

Table 2 shows the scores that the control group attributed to there levels of understanding of the core year 7 objectives.

Table 3 and table 4 show an overall comparison between the students self assessed score and the students' actual score as given by the year 7 tests. In analysing the two sets of results the pupils quantified self assessed score has been used to project a theoretical test score based solely on the individual pupils self assessment as a percentage of the total possible self assessment score of 42. The AFL percentage score was then used to create a possible score attainable in the optional year 7 test based solely on the students' metacognitive self analysis of their understanding of the year 7 core objectives.

Table 3: Overall analysis study group

Student (study)

Total AFL score

% AFL score

metacognitive score

Test score

Difference

1

37

88.1

132

111

21.0

2

30

71.4

107

105

2.0

3

28

66.6

100

96

4.0

4

35

83.3

125

108

17.0

5

33

78.6

118

107

11.0

Standard deviation

7.3

Table 4: Overall analysis control group

Student (control)

Total AFL score

% AFL score

metacognitive score

Test score

Difference

1

42

100

150

127

23.0

2

35

83.3

125

110

15.0

3

29

69

103

121

18.0

4

36

85.7

129

93

36.0

5

34

80.9

121

88

33.0

Standard deviation

8.2

Analysis: Metacognition within topics

In order to attribute any possible effect that the study groups' use of the Chatbot made to their metacognition, it was essential that the individual topics that were covered within the classroom were also analysed. The data collected through the student self assessment of year 7 objectives was again used to aid a comparison between the pupils perceived level of achievement and the actual level of achievement measured by the year 7 optional tests within the topics covered during the period in which the study took place. Both the study and control groups self assessed data was turned into a quantifiable amount by applying the same method as used in the general analysis of results, whereby a smiley face was given 3 points, neutral face 2 points and sad face 1 point. This was then turned into a projected theoretical percentage score attainable in the year 7 optional tests. A comparison was then made between this theoretical score and the students' actual score within that particular topic.

Table 6: Study Group Topic analysis; Geometric reasoning

Study Group

afl score

afl %

Test %

Difference

Geometrical reasoning

 

 

 

 

student 1

3

100.0

88.9

11.1

student 2

3

100.0

100

0.0

student 3

3

100.0

77.8

22.2

student 4

3

100.0

88.9

11.1

student 5

3

100.0

88.9

11.1

Standard Deviation

 

7.0

Table 7: Control Group Topic analysis; Geometrical reasoning

Control Group

afl score

afl %

Test %

Difference

Geometrical reasoning

 

 

 

 

student 1

3.0

100.0

88.9

11.1

student 2

3.0

100.0

77.8

22.2

student 3

2.0

66.7

88.9

22.2

student 4

3.0

100.0

66.7

33.3

student 5

3.0

100.0

55.6

44.4

Standard Deviation

 

11.3

Table 8: Study group topic analysis; Fractions, Decimals and percentages

Study Group

afl score

afl %

Test %

Difference

Fractions decimals and percentages

 

 

 

 

student 1

6.0

100.0

87.5

12.5

student 2

6.0

100.0

100.0

0.0

student 3

4.0

66.7

75.0

8.3

student 4

6.0

100.0

75.0

25.0

student 5

5.0

83.3

87.5

4.2

Standard Deviation

 

8.6

Table 9: Control group topic analysis; Fractions, Decimals and percentages

Control Group

afl score

afl %

Test %

Difference

Fractions decimals and percentages

 

 

 

 

student 1

6.0

100.0

100.0

0.0

student 2

6.0

100.0

62.5

37.5

student 3

4.0

66.7

87.5

20.8

student 4

6.0

100.0

50.0

50.0

student 5

6.0

100.0

62.5

37.5

Standard Deviation

17.3

Analysis of Chatbot learning

As an overall check on the validity of the results that were received it was important that each of the Chatbots that the study group had used were examined to determine whether the study group had taken part in dialogues with the Chatbot

In order to carry this out each of the study groups Chatbot minds were loaded into a fresh install of the Billy Chatbot, this ensured that the individual students' Chatbot mind file was not contaminated by a previously installed mind file. Each of the study groups' Chatbots were then interrogated on their understanding of the topics covered during the study period.

Topics of chat

During the study period, the lessons undertaken by both the study and control groups covered fractions, decimals and percentages and geometrical reasoning. Using the lesson plans created for the lessons, a list of core understanding was created that could be used to interrogate the Chatbots that the study group had used.

conversion between Fractions, decimals and percentages.

Addition of fractions.

Multiplication of fractions.

Parallel and perpendicular line properties.

Properties of regular shapes.

Angles around a point and on a straight line.

Due to the basic capabilities that the Billy Chatbot has, each Chatbot was initially questioned using 'Tell me about…' this resulted in the Chatbot divulging all of its knowledge concerning the topic on which he is being questioned. Once it was ascertained that the Chatbot had a knowledge of the topic, the Chatbot was questioned further using a 'how…', 'what…' style of questioning.

Example shape chat

…

ALEX> how many sides has a square?

FREDDY> a square has four equal sides.

ALEX> how many sides on a triangle?

FREDDY> a triangle has three sides.

ALEX> tell me about a triangle

FREDDY> there are three main types of triangle, scalene, equilateral and isosceles. to find the area of a triangle we multiply its base length by its vertical height. a triangle is a shape. a triangle has three sides. an equilateral triangle has three equal angles. an equilateral triangle has three 60 degree angles.

ALEX> what shape has three sides?

FREDDY> a triangle has three sides.

ALEX> what types of triangle are there?

FREDDY> there are three main types of triangle, scalene, equilateral and isosceles.

ALEX> bye

Example angle chat

…

USER> yes

FREDDY> decisive into a not anything bad contains should!

ALEX> thankyou that was very nice

FREDDY> my wish list very pretty long...

ALEX> lol :-)

FREDDY> i last talked to you today

ALEX> you did indeed

FREDDY> many should music I we shape ever wonderful and about with?

ALEX> shape is not a noun

FREDDY> I'll take your word on that

ALEX> tell me about angles

FREDDY> a square contains four right angles. an equilateral triangle has three equal angles. an equilateral triangle has three 60 degree angles.

ALEX> what shape has four right angles?

FREDDY> a square contains four right angles.

ALEX> tell me about a straight line

FREDDY> I don't have anything to tell you about that!

ALEX> how many degrees on a straightline?

FREDDY> there are 180 degrees on a straight line.

ALEX> well done

FREDDY> well we all there too..lol.

…

Example fraction chat

…

ALEX> tell about a fraction

FREDDY> actually, my girlfriend has a fraction.

ALEX> tell me about a fraction

FREDDY> the top number of a fraction is called the numerator. the bottom number of a fraction is called the denominator. to multiply a fraction we multiply the numerator and the denominator. to turn a fraction into a percentage we multiply by 100. to turn a percentage into a fraction we put the percentage over 100. tell about a fraction.

ALEX> how do I multiply a fraction?

FREDDY> to turn a fraction into a percentage we multiply by 100.

ALEX> how do I add two fractions?

FREDDY> to add two fractions we have to find the lowest common denominator.

ALEX> tell me about the lowest common denominator

FREDDY> I don't have anything to tell you about that!

ALEX> what is the lowest common denominator?

FREDDY> to find the lowest common denominator we list multiples of each denominator to find a common value .

…

Did the study group 'teach' the Chatbot

All of the Chatbots that were used by the study group were questioned on both geometric reasoning and knowledge of fractions, decimals and percentages. In all cases the Chatbots had been taught mathematical facts covering both topics, but in some cases this was to a greater extent than others. This does not in itself detract from the results of the study as the primary focus was to determine the potential benefit of using Chatbot technology as a tool to aid a student's metacognitional development. The fact, that all of the study group, conversed with their Chatbot with reference to the learning undertaken within lessons, was essential to the validity of the study itself.

Is the Chatbot capable of learning mathematics?

From the chats that were undertaken during the analysis of data it became clear that the Chatbot was capable of learning basic mathematical facts during its chats with the study group. It does seem that the Chatbot also displayed a form of understanding of the mathematical facts it had learnt. we can see, from the examples shown, that the Chatbot was able to take two separate facts to create an answer to a question asked of it. This was seen particularly clearly within the chats undertaken concerning shape, all of the Chatbots were able to take known shape facts, ( a square is a shape, a square has four sides) to determine an answer to 'what shape has four sides?'. The Chatbot therefore was capable of creating a bridge, linking the two facts into one piece of knowledge. Beyond this linking of learnt facts, the Chatbot showed no true conceptual understanding of the facts that it had learnt nor did it display an understanding of how to apply its learnt knowledge.

The Chatbot used within the study seems to be permanently fixed at the point of 'potential achievement' as Vygotsky would term it, the next technological step being to give the Chatbot the ability to complete the full learning cycle and turn this 'potential achievement' into 'actual achievement'.

'As computing technology and the underlying language processing software

progresses, we can expect to see potentially exponential growth in the delivered

complexity of Chatbots Already, they have come a long way from their roots in

systems that were more about fun, flirtation or simple 'chat'. We are now

approaching a time where the technologies such as Lingubot can, through extensive

syntactic structures developed for natural language processing and some complex

methodological data structuring, begin to display behaviour that users will interpret

as understanding.'

(kerly ,2006)

Interpretation of Study Analysis

The study groups' standard deviation of the differences between the students assessed score and the actual score achieved, give a clear indication that these students had a good level of understanding of their own 'actual achievement level'. The design of the self assessment form was such, that in order to carry out the task of self assessment the students had to undergo, a certain level of metacognitive activity. The very act of interpreting the language used within the form, matching it with its associated metalanguage and then assessing understanding, being metacognitive in nature. As both the control and study group both underwent this same activity, removes any weight this activity had on the results from the study itself.

Both the study group and control group, did seem to have an awareness of their own level of achievement being able to predict, through self assessment , their 'possible' scores to within, on average,12 marks over the both groups, with The study group predicting to within 7.3 marks and the control group to within 16.6 marks

When examining the effect the study had on the pupils ability to determine level of achievement over all of the core objectives it was found that there was little difference between the study and the control group, the study group predicting with a standard deviation of 7.3 marks and the control to with a standard deviation of 8.2 marks.

The results from the topic analysis however, emphasise the potential that Chatbot technology has in raising students' metacognitive processes, especially those concerned with the ability to self assess understanding. Over both topics the difference in standard deviation of marks received within the optional test and the standard deviation of 'self assessed' mark were significantly different between the control and study group. Within the fractions,decimals and percentages topic the study group achieved a standard deviation of 8.6 % in comparison to the control groups standard deviation of 17.3%, similar results were obtained in analysing the Geometrical reasoning topic, with the study group achieving 7% and the control group 11.3%. Indicating that over the study the Chatbot had an effect on the development of an understanding of the two topics covered, and also on the metacognitive processes involving the topics in question.

Conclusion

This study started with the singular question:

' can AI Chatbots be utilised within the mathematics classroom to develop metacognition? '

In attempting to answer this question, the research carried out has further emphasised the benefits to both the student and the educator in carrying out Assessment for Learning tasks within the classroom context.

The study shows that the present Chatbot technology has a potential use, in enabling pupils to gain a greater understanding of their own learning and subsequently aid the development of their metacognitive processes. The study also demonstrates that AI technology has a place within the tools available to educational practitioners as well as educational researchers. As with all 'new' technology it has its draw backs that, at present, limit its use within the classroom, but these will, as technology progresses, become less of an obstacle to there constructive implementation. At its present stage, Chatbot technology could be used by pupils to create a knowledge bank of information, not just from the field of mathematics but from across the curriculum, that can be easily accessed through the 'chat' interface. Certainly one potential future use of Chatbot technology could be to enable those who through either cultural, social, emotional or physical issues do not take part in normal classroom discourse.

'Research suggests that Face to Face (FTF) interaction in collaborative learning

does not solve the communication problems in Arabic culture due to religious and

cultural factors. It is necessary to think of an alternative that would respect the

general scope of collaborative work (in terms of its multi sided but interactive

effect) but favour women's involvement in sharing and communicating

information with their male fellows. This can be fulfilled through the use of

Computer mediated communication.' (Khan ,2006)

This Research study looked only, at one facet of the use of AI within education, examining the capabilities of Chatbots in respect to a student's metacognition and self assessment accuracy. There are at present many other applications to which this type of technology can be utilised educationally, research is being undertaken examining the role that Chatbots can play as on-line tutors. Although this research is mainly aimed at the higher education tutorial system, a system of a similar nature would be of great benefit within secondary education. Allowing students who may be dissociated from school to have access to tutor discussion at all times.

As such the development of its use within the classroom should be further encouraged and explored.

Future Avenues for research

Before the true potential of Chatbot use within education can be realised a broader study needs to take place that not only uses a larger study and control group, but that utilises technology with fewer limitations. Indeed it was felt during the study that the limitations of the Chatbot technology used would hinder the study itself.

In future research undertaken by the researcher, greater emphasis will be placed on either locating or developing software that is able to create concrete links internally between pieces of learning. The possibility of integrating neural net technology with Chatbot AI technology may enable this level of conceptual understanding.

At a minimum the Chatbot utilised in future research on this topic will need to satisfy the following criteria:

Remain on topic throughout a conversation

Remember the topic being discussed

Be able to logically create blocks of knowledge through the linking of associated knowledge

Have increased NLP skills

Build an internal knowledge base of technical language and its usage

Writing Services

Essay Writing
Service

Find out how the very best essay writing service can help you accomplish more and achieve higher marks today.

Assignment Writing Service

From complicated assignments to tricky tasks, our experts can tackle virtually any question thrown at them.

Dissertation Writing Service

A dissertation (also known as a thesis or research project) is probably the most important piece of work for any student! From full dissertations to individual chapters, we’re on hand to support you.

Coursework Writing Service

Our expert qualified writers can help you get your coursework right first time, every time.

Dissertation Proposal Service

The first step to completing a dissertation is to create a proposal that talks about what you wish to do. Our experts can design suitable methodologies - perfect to help you get started with a dissertation.

Report Writing
Service

Reports for any audience. Perfectly structured, professionally written, and tailored to suit your exact requirements.

Essay Skeleton Answer Service

If you’re just looking for some help to get started on an essay, our outline service provides you with a perfect essay plan.

Marking & Proofreading Service

Not sure if your work is hitting the mark? Struggling to get feedback from your lecturer? Our premium marking service was created just for you - get the feedback you deserve now.

Exam Revision
Service

Exams can be one of the most stressful experiences you’ll ever have! Revision is key, and we’re here to help. With custom created revision notes and exam answers, you’ll never feel underprepared again.