This paper follows up and reports some of the same techniques and results that were published in a recent ACM ITS 10 paper and that they do not cite or discuss. The authors need to explicitly address this in the rebuttal. Apart from that, the reviewers gave generally high scores to this paper, although relatively minor weaknesses were also highlighted.
Double publication concern:
Before I detail the positive and negative aspects that the reviewers
commented on, I need to address several issues related to publication
This paper describes a system that has been published before by a
superset of the current authors (ACM ITS'10).
Towards a teacher-centric approach for multi-touch surfaces in
classrooms. In ACM International Conference on Interactive Tabletops and
Surfaces (ITS '10). ACM, New York, NY, USA, 187-196.
This previous publication also reported on a study. R3 argues that the
Get your grade
or your money back
using our Essay Writing Service!
work presented here provides significant extension on the previous paper.
However, I have been a reviewer for the previous paper and I have several
-The previous paper described already the interaction techniques
(although without the solid design justification from this paper). How is
then possible to present these as the second contribution of this paper
-The previous paper might report results on an version of the experiment
that was already partially reported in the previous paper (they might
have added new groups).
-The authors do NOT cite, mention, or compare the current work with the
work already published. This is particularly surprising because the
authors did not maintain anonymity in their current submission, making it
easy for both reviewers and chairs to find the other paper.
Almost all reviewers commented that this is a very well-written paper. R2
and R3 comment on how they learned something and how there are few
studies that look at the application of multi-touch tabletop technology
to large class scenarios as these. R4 also thinks that the combination of
a formative and a summative evaluation is strong, and the evaluation is
R1 is probably the most critical, citing that a lot of what is presented
in the paper is not really novel, a concern somewhat shared by R3.
R1 and R4 express concerns about the value and origin of the design
requirements. More specifically, R1 does not believe that most of the
design requirements are not obvious or can directly derived from a
general educational perspective. R4 thinks that Design Requirement 4 is
artificial and put there just by convenience. R1 thinks that DR5 is not
really a requirement.
R1 and R2 commented on a possible longitudinal study. I think that a
longitudinal study is not necessary to make a paper like this a
publication-worthy; however, these comments from the reviewers are more
relevant in the light that at least a part of the results from one of the
studies might have been reported before.
Finally there are some concerns by R2 on the lack of discussion of how
the data was collected and analyzed, of how the results would generalize
to other tasks (there are many different tasks that can take place in
classrooms), and how some conclusions are not traceable to the reported
data (Visiting students is a burden?).
It should also be noted that the paper is almost half a page over the
Points to address in the rebuttal:
-the issue of double publication/the novelty of the contributions if
considered against the ITS paper.
-the relationship between the data reported in both papers
-the lack of a discussion on the data collection and analysis methods
-the lack of a discussion on the generalizability of the results
------------------------ Submission 346, Review 4 ------------------------
4 (Probably accept: I would argue for accepting this paper.)
This paper describes two studies, one formative and one summative, of
Always on Time
Marked to Standard
user interface techniques geared toward supporting the use of multi-touch
tabletops in school classrooms. Through analysis of video data of
teacher-student interaction and application use, the strengths and
weaknesses of the different features designed into the classroom
management system are presented.
This is a well-written paper with good quality prose. I appreciate the
presentation of material, and the authors clear use of both formative and
summative evaluation to assess system design. Together these studies
provide insight into how best to design tabletop interfaces.
I am also aware of no work that fully replicates the work presented in
this paper, but I am not an expert in this field. I do know that many
researchers have expressed an interest in applying tabletop technology to
the classroom environment. I hope that some domain experts are among the
reviewers for this paper to ensure that the contributions outlined here
are novel. Because I am not aware of significant prior art in this
field, I assume that the work is novel and evaluate it as such.
While the paper is well-written and significant analysis was clearly done
by the authors, there are components of this paper that seemed less than
impressive to me. It seems to me that the use of tools observed in the
final design could easily have been predicted based on an understanding
of the teaching process. Teachers' goal in assessing student learning
during a classroom activity, the _reason_ they wander around the room, is
to ensure that groups are staying on task, making progress toward a
solution, and that all group members are participating. Dealing with
issues like occlusion of material seem immaterial and childish in a
design that adapts in an informed way to teachers' goal in classroom
management. It almost seems some features, particularly DR4, was
inserted because it was assumed it should be important, rather than
arising naturally from an understanding of users. Even in the formative
study, it seemed that this feature was promoted by the researchers, as
opposed to being an issue that arose naturally from teachers' goal. We
saw occlusion, asked teachers about it (Prompting them, essentially) and
they said, "Oh, yeah, that'd be nice." It seems almost that some of the
design features were promoted by multi-touch researchers pushing their
view of technology, and this researcher bias persists in classroom
layout. Consider Figure 1. Is it really a good idea for the teacher
table to be positioned in such a way that teachers need to turn their
backs on a class? One thing any elementary of junior high school teacher
will tell you is that classroom control is all about constant monitoring
of the physical classroom environment. Hunched over a desk facing away
from students is not a natural orientation for teachers.
While I do have minor concerns with the work, as outlined in the previous
paragraph, I think the evaluation of a deployed solution using teachers
and students in realistic settings was largely effective in evaluating
the use of tabletop technology for classroom learning. I still don't
believe that there exists much benefit to this technology over
paper-based interaction in classrooms. I believe that better solutions
can be designed that are less expensive and more versatile in terms of
monitoring and sharing information than the electronic tabletop interface
described, and that these solutions need not have the inherent drawbacks
of forcing students to work with virtual artifacts when tangible
artifacts are more effective. However, the goal of this paper wasn't to
argue that tabletops were a panacea for the education system. Given that
we want to create this artificial market for tabletops in classroom
(analogous to the artificial market that has been created for
smartboards) I can conclude that this design study is a useful step in
------------------------ Submission 346, Review 1 ------------------------
Reviewer: Program Review Committee Member
3 (Borderline: Overall I would not argue for accepting this paper.)
This Essay is
a Student's Work
This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.Examples of our work
2 (Passing Knowledge)
This paper has a lot in it that is really not overly novel. For
instance, the following design requirements that were elicited from
DR1: Lots of computer assisted learning software has this as a principle
feature. Here's a naive version that shows up when I first google for
related examples http://www.neuber.com/usermonitor/index.html, but there
are plenty of examples of similar systems that allow for monitoring and
control. I don't think this design requirement is novel.
DR2: Same thing as DR1. Perhaps if the design requirement touched on
novel aspects of the table or novel methods of interaction that were
needed for instructors to control views that would be good, but as is
it's not a very significant revelation.
DR4: Occlusion and orientation of artifacts on tables is a well known
DR5: Interesting, but I would like to know more. Would teachers actually
take the time to review and trace back through student activities to
build individual evaluations? The evaluation suggested that this wasn't
very popular, and that fits in line with my understanding of the time
pressures teachers are in during the classroom.
The evaluation is ok, but I would have liked to see (of course), a
longitudinal study. Table top computing is really new, and there is a
huge novely effect. That teachers would rate it highly is not a
surprise, it's pretty cool stuff. But would they rate it so highly after
it is deployed in their classroom for a year? And compared to other
IT-based classroom management tools, is it really much better? What's
novel about this system that makes it better? This is only alluded to in
section 4.3 ("Two teachers, who had prior...") but the authors don't dig
into why their system is better. How many other teachers used classroom
management tools and didn't find this one better?
------------------------ Submission 346, Review 2 ------------------------
Reviewer: Program Review Committee Member
4 (Probably accept: I would argue for accepting this paper.)
This paper investigates interaction techniques that will effectively
support teachers within a tabletop classroom. It reports on a pre-design
study that identified potential issues from the teachers' part and
provided the design requirements; exhibits the design solutions that
addressed these requirements; and provides the reader with the findings
from an evaluation study.
Overall this paper is well written; it is clear, easy to follow and well
argumented. The design process described is robust and the findings
reported are quite interesting. It is true that there are very few
studies reporting on an ecology of multi-touch devices within a classroom
environment and for this, the contribution of this work is valuable for
HCI. On a personal note, I would like to see similar studies carried out
in real classrooms as I would suspect there is a whole range of other
challenges and difficulties for the teachers involved. Still, this work
identifies some real issues of management and control that can occur
within a real classroom.
However, there are a few issues to considerate. Firstly,it would be
useful if the authors added some more details about their data collection
and analysis. While the method is briefly stated, there is no mention of
any detail on how the data was collected and/or analysed - that is both
for the observational and the questionnaire data.
Secondly, the 'Related Work' section is somewhat confusing. Usually
related work is preceding and is used to frame the findings reported in
the paper, while a discussion section is more appropriate after the
findings. What the authors provide here is something between a discussion
and a related work section and some disambiguation is necessary.
Thirdly -as the authors briefly mention in the conclusion, it would be
interesting to see if the same issues of design requirements and/or
implementation emerge in the context of a different task/activity. Also,
a long-term study would be advisable to better understand the effects of
the tabletops in the children's engagement and determine possible false
indications due to the novelty effect.
Some other issues to address: a)why the authors state on page 6 (Design
Solutions) that visiting student tables was a burden? The teachers do not
seem to mention this in their reports.
b) on the teachers' feedback section, it would be nice to know why this
one teacher did not rate the system positively.
c)references were not alphabetised,
d) paper exceeded the 18-page limit
------------------------ Submission 346, Review 3 ------------------------
Reviewer: Program Review Committee Member
5 (Definite accept: I would argue strongly for accepting this paper.)
I thoroughly enjoyed reading this submission. It was well-written, and I
feel like I really learned something!
The authors address the problem of how to design a meaningful
multi-display environment for classroom teaching. Their particular focus
in this context is multi-touch table-based interfaces where students
collaborate with one another. The goal of the work appears to be to
explore how to design an instructor-centered counterpart to this system
in the classroom. The authors present a formative study where the
evaluate the potential for the instructor's system, actually go ahead and
design the system (providing considerable detail on this), and then
finally evaluating the system as a whole with several classrooms. The
contribution of this work is a set of design principles for a
teacher-centric system for monitoring and managing tabletop collaboration
This piece of work extends some work that was published at the ITS 2010
conference, and adds substantial new content. In particular, the ideas
coming out of the study from ITS (and presented here as the "preliminary
study") have been (appropriately) reframed here as design
recommendations, and used to iterate on the existing design. This
discussion of the iterations and the design recommendations are
interesting. While the ideas are not groundbreaking, they provide a
logical "next step" in this particular application space that are
appropriate. This submission also provides an additional study that is
used to evaluate the subsequent design, providing additional observations
on how such a system might be employed in the real world.
This is an interesting, comprehensive piece of work. It raised some
interesting questions about the pedagogical uses of this kind of tool.
What are the learning objectives that can be achieved using such a tool
in a classroom? In some sense, this is merely a platform (so in theory,
facilitates "anything"); however, its design implies a certain way of
interacting with information and classmates (along with the teacher):
what implications does this have for learning? I imagine also that
teachers enjoy walking around a classroom while group activity is going
on -- was this considered in the design? Would an "iPad"-like version of
the teacher tool be useful? And, how would it be used/used differently?