This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.
This paper presents a new approach to specifying critics for domain-specific visual language tools using a visual and template-based approach. Predicate, rule-based, knowledge-based and direct programming are among many approaches that have been used to design and implement critics for computer-based tools. However, there is still little agreement on how critics should be specified and little research on tools to author and implement critics in complex environments. In this paper we discuss our approach for specifying critics for domain-specific visual language tools. This allows target end-user tool developers to design and implement critics efficiently in a natural manner. We describe a survey that we conducted to evaluate our new approach and the Cognitive Dimensions approach that was applied in the survey questionnaire design. Survey results are discussed along with the strengths and weaknesses of our approach and suggestions for future research.
Categories and Subject Descriptors
D.2.2 [Software Engineering]: Design Tools and Techniques - evolutionary prototyping.
critic, critiquing, critic specification, critic authoring template.
Much research has been devoted to critic tools (or critiquing systems). ArgoUML , Design Evaluator , Java Critiquer  and many others are examples of computer-based critics. These critic tools produce critiques (or critic feedback) that are specific to their problem domains. The use and context of critics varies from one domain to another. To date, critics have been applied in a wide variety of domains including education, medicine, design, programming and software engineering [9, 14-18, 22]. In general, critic tools provide knowledge support to users who lack specific pieces of knowledge about their problem or solution domains. Furthermore, they detect potential problems; give advice and alternative solutions; and possibly provide automated or semi-automated design improvements for users.
Inspired by the existing critic tools work, we made an attempt to apply similar ideas to our meta-modeling tools, called Marama. Marama is a meta-tool implemented as set of Eclipse-plugins and includes meta-tools as well as modeling tools . Most of the existing critic tools that we reviewed are not developed within the environment of a meta-modeling tool. Our meta-tools are used to generate complex visual modeling tools, and these modeling tools could benefit from the addition of various critics. Thus, we wanted to extend our Marama meta-tools by embedding a critic design and generation component. The main purpose of our work is to assist end-user tool developers to specify and generate critics efficiently and easily.
In this paper we present the background and motivation of our research. We then describe the design and implementation of our approach for specifying DSVL tool critics. Next, we describe an end-user survey we conducted to validate our approach. Finally we conclude with contributions of our research and suggest further research directions.
BACKGROUND AND MOTIVATION
In earlier work we developed a critic taxonomy, shown in Figure 1, based on our review of related literature concerning critics [1, 2, 6, 7, 9, 14-18, 21, 22]. Our taxonomy has eight groups and elements that were adapted to meet specific needs. Our objective in undertaking the critic taxonomy development was to support us in designing and developing our own design critics for domain-specific visual language (DSVL) tools developed using our Marama meta-tool . Several elements from our taxonomy are considered in the critic development approach described here, and these will be referenced in the following sections.
We describe in more detail two groups from the taxonomy in particular which reflect the motivation for our research work: critic realization approach and critic rule authoring. The critic realization approach group concerns the approach taken to specification and design of critics. We identified the following as the possible approaches for implementing critics: 1) predicate, 2) rule-based, 3) knowledge-based, 4) pattern-matching, 5)
Critic Groups and Elements
Modes of Critic Feedback
Critic Rule Authoring
Critic Realization Approach
Types of Critic Feedback
Types of Critic
Insert new critic rule
Modify critic rule
3-Dimension (3D) Visualization
Delete critic rule
Enable / disable critic rule
Examples (or precedent)
Critic rule authoring facility
Object constraint language (OCL) expression
Design PatternFigure 1: A Proposed Critic Taxonomy.
Table 1: Critic realization approach
requires advanced programming skills to construct knowledge base
critic customization is not easy
requires expertise and knowledge of construction of sets of IF-THEN rules
need to write rules for all problems before deployment to users
critic customization is not easy
requires good programming knowledge and skills
restricted to a particular language so hard to reuse across platforms
critic customization is not easy
requires deep understanding of pattern-matching methods
critic customization is not easy
requires good comprehension of predicate logic methods
critic customization is not easy
Object constraint language (OCL)
demands deep understanding of OCL syntax
critic customization is not easy
programming code and 6) object constraint language (OCL). Table 1 shows barriers to adoption of each approach in designing critics.
Several existing critic systems have applied a knowledge-based (KB) approach for specifying and implementing critics (e.g., IDEA  and Design Evaluator ). The knowledge base represents the most important and usually complex component of a knowledge-based system . Designing critics using a KB approach normally requires advanced programming skills in order to construct the knowledge base . The knowledge base is normally developed by experts who make their domain knowledge accessible.
For instance, the IDEA (Interactive DEsign Assistant) tool  produces design pattern critics implemented with Prolog rules that are directly integrated with a knowledge base. Bergenti and Poggi stated the knowledge base of IDEA is comprised of a set of design rules, corresponding critics, and a set of consolidation rules . The rules for creating the pattern-specific critics are not straightforward as they require a high-level of understanding of a design patterns and detailed knowledge of Prolog and knowledge base structures .
Critics implemented with a rule-based approach consist of a condition and an action. If the condition is true, then the action is performed. Actions can include suggestions, explanations, argumentations, messages or precedents of problems. For instance, ABCDE-Critic  uses rule-based expression to specify critics that critique UML class diagram-based designs. The critic tool invokes critics when a condition clause is found to be true in the current design, warning the user that the design may have a problem . Although it tends to be easy to use and understand the rules concept, it requires reasonably deep programming and domain knowledge to construct the set of IF-THEN rules for all possible problems in a particular domain.
Many critics to date have been realized through the use of programming code. For instance, critics in Argo/UML  are coded as Java classes. Class Critic identifies several methods that can be overridden to define and alter a new critic. Each critic's constructor identifies the headline, problem description, and relevant decision categories . The problem for tools that specify critics via programming code is that all they are bound to a particular code or design language, losing the opportunity to be reused across languages. Good programming knowledge and skills are needed to specify the critics.
Another approach to realizing critics is to use pattern-matching. In order to check whether things have a desired structure, or to replace the matching part with something else, we can adopt a pattern matching approach . Pattern matching normally consists of left-hand side and right-hand side rules. Various pattern-matching techniques can be applied, thus, it is essential to have a good understanding of pattern-matching methods to represent critics for all possible situations. For instance, the Java Critiquer tool performs automatic critiquing using a pattern matching approach . The left-hand side of a rule is a LMX pattern (Language for Mapping XML). The right-hand side of a rule is a critique. In the Java Critiquer, the pattern is a JavaML pattern for matching JavaML code generated from the Java parser. When a pattern is matched, its matching critique is added directly below the problematic Java source code .
A predicate logic approach can be used to implement critics. According to , predicate logic is based on the idea that sentences can express relationships between objects as well as qualities and attributes of such objects. The terms of the predicate are represented by the objects. The use of terms lets a predicate express a relationship about various objects rather than just a simple object . Furthermore, predicates can be applied to represent an action or an action relationship between two objects . Designing critics using this approach requires a good comprehension of predicate logic to use it to model the problems of a certain domain. An example of a tool that applies the predicate approach is the Design Evaluator . The evaluation layer in Design Evaluator compares sketches with predicates that represent design rules and generates critiques when rules recognize a pattern in the design. It compares recognized spatial information with each rule. If it finds a rule violation it generates a design critique to be displayed in the visualization layer [ref]. The rules are coded as Lisp predicates that apply to the design objects.
OCL expressions are increasingly being used to specify critics. The Object Constraint Language (OCL) is "a declarative language for describing rules that apply to Unified Modeling Language (UML) models" . A research of model checking by , demonstrated the use of OCL to express constraints via a simple domain-specific language (DSL) called Class Diagrams (CD). As  argues, OCL needs extensions to support additional elements such as the severity of a constraint attached to constraints. They classified the severity of a constraint as an error, a warning or a critic . In their CD approach they show how a critic can be specified using an OCL expression. However, doing this demands deep understanding of OCL syntax to be able to specify meaningful critics.
In general, the customization of critics is not easy because it requires overall comprehension of the "programming games" of the approach employed and also knowledge of the critic domain. All of the approaches above suffer from this problem, which acts as a major barrier for end user tool developers.
The other group we highlight in our taxonomy is critic rule authoring which describes typical tasks undertaken when implementing critics. Elements in this group involve: insert new critic rule, modify critic rule, delete critic rule, enable and disable critic rule, and critic rule authoring facility. Critic rules are one of the important components in building critics. In general, critics are specified by a single rule or groups of rules (or procedures) to evaluate different aspects of a product or design , no matter what the actual implementation method is. Thus, a critic development must involve, at some level, the authoring of critic rules. As [14, 22] argue, critic rules are normally written in advance by system designers to develop a critic system and it is hard for end users to modify existing rules or add new critic rules after a critic system is deployed. However, as [9, 22] argue, critiquing may need to be adjusted depending on various situations. Furthermore,  emphasizes that users should not be required to have comprehensive programming knowledge in order to perform the modification of critic rules. For these reasons it is important to allow users to understand critic rules and be able to modify and expand the rules by authoring new rules to incorporate in critic systems. In addition, some tools allow users to enable and disable critics via menu configuration. A disabled critic will thus never be executed until the user enables it, and they are unlikely to do so without understanding it. In general, the capability for rule authoring is to enable end-user designers to construct, tailor and store their own critic rules. A rule authoring facility thus allows end-user tool developers/customizers to contribute to the system's feedback process .
The need to specify critics in a simple way using an easy to use, high-level language is the motivation for our research. We see the opportunity to apply domain specific visual language (DSVL) approaches to reduce end-user barriers. DSVLs are graphical notations specially devised for specific needs and knowledge . The languages allow anyone who is a domain expert to use the visual language as an application development tool for the domain . The following work has motivated our research in specifying critics for DSVL tools: (i) various application domains of critic tools (or critiquing systems), (ii) metamodelling tools that have constraint evaluation and (iii) business rule templates from the business process domain.
VISUAL AND TEMPLATE-BASED APPROACH
We propose a visual and template-based approach to support end-user developers in critic specification tasks. We have developed and applied this in the context of our Marama meta-tool , which provides a range of other facilities to assist end-users to develop modelling tools, but the concept is applicable to other types of diagrammatic tool. In this section, we describe three main parts to our critic development approach: 1) a visual critic definer, 2) a template-based approach and 3) critic execution-examples.
Visual Critic Definer
The main underlying idea in our approach is to use information expressed in a meta-diagram (in our case the Marama meta-model diagram) as input for critics to be realized in a diagram (in our case a Marama diagram in the realized modeling tool specified by the meta-model). It is important to mention that our approach is only minimally dependent on the notation used in the meta-diagram. The Marama meta-model diagram is expressed using an Extended Entity Relationship (EER) notation which specifies entities and relationships, together with their attributes . If a richer notation is used, more information can be extracted from the meta-model diagram and, thus, be used for specifying critics and checking user diagrams. Figure 2 shows an example of a meta-model for the health care planning model (CPM) tool from a medical domain and the realized tool in use modeling a care plan for diabetes management .
Figure 2: Health Care Planning Model tool meta-model (top) and in use (bottom) 
Currently Marama-based tools are developed via three main editors: the meta-model definer, to specify a tool's meta-model; the shape designer, to design the tool's visual notational elements; and the viewtype definer, to specify mappings of meta-model elements to visual representations. Due to the argument made by Fischer et al. that critics should be considered as embedded systems rather than as stand-alone components , we have created a critic definer to support the critic specification tasks. Once a tool is equipped with sufficient information, the end-user tool developer then can specify the required critics for that tool via the critic definer. Figure 3 shows the notation used in the visual critic definer editor.
The visual critic definer editor has three main elements: CriticShape, CriticFeedbackShape, and Operator, and three connectors: CriticFeedbackConn, CriticDependencyLink, and OperatorConn. The CriticShape (rounded square shape) allows a target end-user developer (or tool developer) to specify critic(s), whereas the CriticFeedbackShape (oval shape) is used to specify the feedback for each defined critic. The Operator (AND, OR, and XOR) supports the creation of composite/compound critics. The relationship between critic and feedback is supported by the CriticFeedbackConn. In a case where one critic is dependent to another critic, a CriticDependencyLink is used for visual representation. The OperatorConn links critics with the logical operator (AND, OR, and XOR) to form a composite critic. This allows complex critics to be readily built from simpler parts.
The main difference between our approach and other approaches for specifying critics is that we provide end-user developers with a critic authoring template. End-user developers are able to specify and author their own critic(s) using this template-based approach. We were inspired by the business rule (BR) templates approach , adapting this concept for use in our critic authoring templates. We chose this approach as to enable end-user developers with limited programming capability to be able to specify critics for their domain-specific visual language (DSVL) tools. Our reasoning was that as BR had been proved useful in an end user oriented business rule specification domain , it would be useful for our own domain which had similar levels of complexity.
Figure 3: Visual critic definer editor
Our critic authoring templates are applied to a target DSVL tool's meta-model to review its target model instances. We also provide critic authoring guidelines to assist the end-user developers in specifying a critic. The critic specification is defined by selecting a CriticShape in the visual critic editor as shown in Figure 4 (top). The CriticShape is associated with a form-based interface designed to ease the task of specifying and authoring critics. Figure 4 (bottom) shows the associated Critic Construction View interface. The target end-user developers specify their critics by selecting from available templates provided in Critic Construction View interface and completing the form to enter required information.
Figure 4: CriticShape (top) associated with Critic Construction View interface (bottom)
Table 2: Constraint and Action Assertion Templates 
<entity> must have | may have a [unique] <attributeTerm>
<attributeTerm1>must be | may be <relationalOperator> <value> | <attributeTerm2>
[<cardinality>]<entity1> is a/an <role> of [<cardinality>]<entity2>
[<cardinality>]<entity1> is associated with [<cardinality>]<entity2>
<entity1> must have | may have [<cardinality>]<entity2>
<entity1> is a/an <entity2>
When <event> [if <condition>] then <action>
The critic authoring templates support three types of template: attribute constraint templates, relationship constraint templates, and action assertion templates. Attribute constraint templates are used to specify essential properties around uniqueness, optionality (null), and value check of an entity's attributes . The relationship constraint templates assert relationship type, cardinality and role constraints s of each entity participating in a particular relationship . Action assertion templates specify an action to be activated on the occurrence of a certain event or on the satisfaction of certain conditions . Table 2 describes the BR templates that we have adapted into our critic authoring templates.
However, we do not limit our critic authoring templates to the ones proposed in the BR approach. We wanted end-user tool developers to be able to specify their own critics. Hence, in a case where the available template does not support the desired critic specification, we allow the developer to construct a new critic template via a Critic Template editor, shown in Figure 5. The end-user developer initially needs to construct the new critic statement/phrase. Based on the critic statement, the developer selects the necessary properties to form a new critic template that represents the new critic statement that has been defined. Our critic authoring templates are not as highly expressive as natural language rule statements, but provide sufficient expressiveness to allow end-user developers to understand, modify and possibly author critic rule expressions with little support from expert tool developers. The critic rule templates also leverage the rich meta-model diagram facilities, allowing experienced end-user developers to author moderately complex domain specific templates. After specification, new critic templates are listed in the available templates and can be used to specify critics. Thus, the available templates list can be expanded according to the new critic templates created in the critic template editor.
Figure 5 : Critic Template Editor
Once the critic(s) has been defined in the visual critic definer editor, the next task is to specify feedback for the defined critic(s). This is done via the CriticFeedbackShape. The CriticFeedbackShape is also associated with a form-based interface, the. Critic Feedback View, shown in Figure 6.
Figure 6: CriticFeedbackShape associated with Critic feedback view interface.
The end-user developer needs to specify the necessary action to resolve the critic(s) defined for that DSVL tool. The critic feedback view has the following properties: (i) critique strategies that determine the execution mode of the critic. This can be either active or passive. An active critic will monitor continuously a user's tasks and warns the user as soon as a critic is violated and then provides feedback (a critique). A passive critic only works when a user asks explicitly to check for a critic violation. (ii) modalities of critiques involve the presentation of the critique. This can be textual, graphical or a combination of both; iii) an explanation represents a reason/justification of a critique; iv) a suggestion indicates an action to resolve the critic violation; and v) a critique message specifies a textual message that is displayed for each critic that has been defined.
Once a critic and feedback are specified and defined, these two elements are linked to indicate that a critic comes with a fix action. Although  stated a critic does not necessarily solve a user's problems, in our approach we expect the end-user developers to indicate a fix action, where possible, for each critic defined for their DSVL tool.
The use of action assertion templates, critic dependencies and critic combination operators is described in the following section. All critics and feedbacks that have been defined are saved in an XML format in the meta-tool's repository. Critics and feedbacks are instantiated when the tool is started by a user.
Figure 7: Examples of unit critics using the (top, centre) attribute constraint template and (bottom) relationship constraint template.
To illustrate our critic authoring tool in action, we show several examples of critics and feedbacks defined using it. As mentioned previously, a critic specification is dependent on the information expressed in the tool's meta-model. We use the meta-model for a health care planning model tool shown in Figure 2 as an example against which we will specify critic(s). In Figure 7, we see three critics have been defined. At top a null constraint has been specified for a CarePlan entity. Here an attribute constraint template has been selected in the CriticConstructionView editor (the last of the one's shown in the drop down box in Figure 4) and the CarePlan entity and CarePlanName attribute selected as the entity and attribute term respectively. At bottom is a cardinality constraint on the relationship between the CarePlan and PerfomanceMetric entities, specified in a similar manner. All properties in the tool's meta-model are available in the Critic Construction View interface, selectable via drop down menus. Feedback actions for each critic have also been specified. Both have been specified as Active critics with appropriate explanation and fix messages to resolve them. Critics are generated automatically after the end-user developer completes the required properties for each critic. The execution one of the critics specified in Figure 7 is shown in Figure 8. Presenting the critique message and fix action are based on properties that were designed in the Critic Feedback View interface.
Figure 8: The execution of a critic and a fix action for the not null name critic specified in Figure 7(top).
Figure 9: Critics specified in critic definer editor (bottom) based on the meta-model of SimplifiedMaramaEML tool defined in the meta-model editor (top).
Figure 9 shows additional critic examples, specified against another tool, MaramaEML, a business process specification tool . The bottom-most critic in Figure 9 is an example of an action assertion template in use. Here the tool developer wants the service entity to have no more than four operations. A critic can be specified for this case by defining the relevant properties for event, condition and action in an action assertion template as shown in Figure 10. Here the event is the creation of an association link, the condition is the cardinality is greater than 4 and the action is to delete the new association. When the user runs the tool, a critique will be displayed if the event occurs to warn the user, followed by execution of the action (see Figure 11).
Figure 10: An example of a critic specified using an action assertion template
Figure 11: Action assertion critic execution after the trigger event occurs: a critique is displayed to warn the user
The critics described in Figure 7 are unit/single critics, where the end-user developer constructs one critic at a time, based on one preference. They can also construct complex (or composite) critics which extend the expressive power, while still retaining the relative simplicity of the BR template approach. A complex critic is a critic that has multiple features that need to be measured. This can be done using logical operators AND, OR and XOR. This approach allows users to specify complex critics by building them from parts and also facilitates reuse of simple critic parts.
The top most critic in Figure 9 is a complex critic, where two critic conditions, in this case two name uniqueness constraints, have been ORed to share a common feedback element. A composite critic, such as the centremost one in Figure 9, supports the situation where one critic might be dependent on another, represented visually with a CriticDependencyLink. In the case in Figure 9 the critic: "EMLService must have a unique name" is dependent on a critic: "EMLService name must not null". This means that the unique name critic is only checked if the service name is not null. Otherwise it will run the unique name critic. The execution of a composite critic using an operator (as defined in Figure 9) is shown in Figure 12, where an operation has been defined with an already existing name.
Figure 12: Feedback of a composite critic using the logical operator OR (top) and fix action for this critic (bottom).
Figure 13: An example of a passive critic.
End-user tool developers are provided with the option of having active or passive critics as described earlier. Figure 13 shows an example of passive critic. Only when a user asks for a check of critic rule violations does it display the critiques of rules that have been violated. It would be straightforward to extend our approach to, for example, integrate with the Eclipse problem marker view to provide an active-passive combination as well.
To evaluate our approach we conducted a user evaluation with ten volunteer researchers and students who had basic background knowledge of the Marama meta-tools and who were interested in modeling and the development of modeling tools to support their work. Four of the subjects were computer science researchers, who have used the Marama meta-tool to develop tools for their research work. Another 6 respondents were postgraduate computer science students who had taken a course in which Marama had been introduced and had as result done an assignment using the Marama meta-tool. The objective of this tool evaluation was to assess how easy and useful it is to specify critics using the visual and template-based approach.
Initially, the participants were given a description on how to use the visual critic definer editor and the functions involved with it. We then asked participants to perform several tasks on the Marama Critic Definer with supervision and then fill out a questionnaire at their own pace without supervision. The questionnaire was in two parts. The first focused on standard usability questions, such as ease of use and effectiveness. For the second part, we adapted the questionnaire designed by  based on the Cognitive Dimensions of Notations (CDs) framework . This provided questions targeted at each of the cognitive dimensions as we were interested in the tradeoffs amongst those dimensions that respondents observed. For both parts respondents answered on a 5 point Lickert scale (1=strongly disagree, 5=strongly agree). The results are shown in Figures 14 and 15.
Figure 14: Usability responses. Y axis is number of respondents for a given response
The usability responses shown in Figure 14 were highly positive. In all questions by far the majority of participants answered that they agreed or strongly agreed, indicating the tool had strong appeal, and was perceived to be highly usable, useful and effective by our target end users, tool developers.
Figure 15 shows the response to questions concerning each of the cognitive dimensions. In the following, we discuss each of these in more detail
Figure 24: CD questionnaire response. Y axis is number of respondents for a given response
This CD indicates the ability to view various aspects of the tool easily. Nine out of total 10 respondents answered that it is easy to see various parts of the tool. The Marama Critic Definer view shows two simple visual notations to represent critic (i.e. CriticShape) and critic feedback (i.e. CriticFeedbackShape), and two connectors to show the link between critic and critic feedback (i.e. CriticFeedbackLink), and dependency between critics (CriticDependencyLink). The CriticShape and CriticFeedback Shape are associated with form-based interfaces to assist the user in specifying a critic and a critic feedback. The only respondent who doubted the easiness to see various parts of the tool commented that due to the lack understanding of meta-tool concept and as a novice user it is hard to see the function of various parts of the tool.
Viscosity reflects a design's resistance to change. Nine respondents said that it was easy to make changes. Only one respondent answered undecided. The user can easily change critics and critic feedbacks that have been defined in the Marama Critic Definer view.
Diffuseness refers to the verbosity of language, i.e. the number of symbols required to express the meaning. Eight respondents answered that the notation is succinct and not long-winded. The respondents comment that the notation is a straight forward representation of a critic and a critic feedback, as well as the connectors that link them. A critic is defined via three templates provided in the critic construction interface. Thus, a user only defines a critic based on the selected template. Whereas to specify a critic feedback only requires the user to defines five properties: critiquing strategies, modes of critiques, explanation, suggestion, and critique message. Two respondents replied undecided to this element.
Hard Mental Operations
This dimension reflects the degree of demand on cognitive resources. Three respondents disagreed, four respondents agreed (3 agree and one strongly agree) and three were undecided as to whether using the tool required hard mental effort. The four respondents who agreed claimed that they needed to concentrate and think carefully to use the critic templates for specifying a critic. This may be because users were unfamiliar with the critic authoring templates.
Error proneness refers to the ability of the tool to induce 'careless mistakes'. Five respondents disagreed, three respondents agreed and two were undecided as to whether the tool was likely to induce mistakes. This dimension has a similar issue as did the hard mental operations dimension. The respondents that answered it is easy to make mistakes raised the issue that unfamiliarity with the templates can cause users to make mistakes in specifying critics. However, five respondents found low error proneness as the notation is very straightforward and supported by a form-based interfaces which is familiar to most users.
Closeness of Mapping
This dimension reflects the closeness of the representation to the domain it describes. All of the respondents agreed that the Marama Critic definer view provides a notation that closely related to the domain. The critic definition closely relates to the critic statement/phrase that the user specified based on the available templates. The specification for a critic feedback too is straightforward by just clicking on the necessary options and adding the explanation and a critique message.
Role expressiveness indicates that the relationships among components should be obvious. Seven respondents answered it is easy to tell what each part is for when reading the notation. Only one respondent disagreed and two respondents were undecided. In the Marama Critic definer view, it is obvious how to specify a critic and a critic feedback because it only involves two simple notational elements with each associated with a form-based interface.
This dimension assesses the existence of hidden links among parts of the tool. Eight respondents said that the dependencies are visible and two respondents are undecided. Hidden dependencies are primarily between the visual critic definer view and the form based template views. Moody argues that this type of hierarchical dependency is of positive benefit in his Principal of Complexity Management .
Progressive evaluation indicates the ability to test code as it is being developed. Nine respondents answered it is easy to stop and check work progress. The Marama Critic definer view allows the critic and critic feedback specifications to be evaluated at any stage. Partially completed critics and feedbacks for a developed Marama tool can be executed as well. Critics and Feedbacks properties can easily be edited and any new changes will take effect during the model execution of the tool.
This dimension reflects the order of steps that a user must execute to achieve a specific outcome. All of the respondents agreed that there are no premature commitments in the Marama Critic Definer view. The user can freely specify a critic using any templates (attribute, relationship or action assertion). However, the user needs to define a critic first before a critic feedback can be specified and linked with the defined critic. The user does not need to have a complete set of critics and critic feedbacks to be specified in the Marama Critic definer view. The user can add a critic as well as the critic feedback for the Marama tool incrementally as he/she encounter new critics.
We also provided the respondents with an open question and a space for them to write comments about improving the critic authoring tool. In general, the comments/feedbacks suggested that specifying critics visually and via template-based style are simple and effective. Issues that raised by some respondents to improve the tool are:
Consider using more artificial intelligence (AI) techniques;
Transform the critic templates into visual entities;
Give a more detailed explanation of templates;
Consider coloring to differentiate different types of critic;
Add a more explicit visual representation of the relation between critics and the tool's meta-model elements;
Consider highlighting the design item that triggered a critic;
Add feedback information into a visual critic element with connection and layout automatically generated.
The limitations observed in our tool were all relatively minor and readily correctable.
DISCUSSION AND CONCLUSIONS
We have described our critic development approach to support end-user tool developer to specify critics in a simple way. We introduced our proposed critic taxonomy that has supported and guided us in our critic tool development. We have demonstrated a visual and template-based style to support the task of end user specification and realisation of critics using examples from Marama-based DSVL tools. We also described a survey that we conducted to assess our visual critic tool. The survey results demonstrated that for most target end users of our meta-tool our approach of using a visual design notation with template-based details appears to be useful in assisting these end-user tool developers in the critic specification task. Our approach also nicely complements the other components of our Marama meta-tools and is integrated with these.
Referring back to our critic taxonomy in Figure 1, our visual critic tool does not include all the elements in the taxonomy group. However, our critic development approach does support the eight groups identified in the taxonomy. Our visual critic tool could potentially be used for any domain-specific visual language tools with appropriate integration and code generation support. So far, we have used it with several Marama-based tools to employ the Marama critic definer view. We have used analytical critiquing rather than comparative critiquing in our approach as this does not require complete knowledge of the domain problems and critics can be built incrementally throughout the design process . Currently our mode of critic feedback uses a textual style for the explanation property and a graphical style to support the suggestion properties. The types of critic that are supported by our visual critic tool are: structural critics, correctness critics, completeness critics and consistency critics.
The contributions of our research are in providing a visual way to express critics for DSVL tools. A notational representation is offered to end-user tool developers to specify critics for their DSVL tools without the need to have a deep technical knowledge of critic construction. Our research provides a critic authoring template-based approach which is much easier and quicker compared to other existing approaches in specifying critics. It is easy for end-user tool developers to customize critics and introduce a new critic template as we provide them with a critic authoring guideline and critic template editor. We have demonstrated a proof-of-concept of our approach by implementing it within the Marama meta-tool and evaluated the resulting prototype visual critic tool using a Cognitive Dimension (CDs) approach. In addition, the proposed critic taxonomy has supported us in identifying the needs of our own visual critic authoring research and tool.
Several areas for further research are to extend the critic capabilities by allowing the critics to check the tool's metamodel elements (MetaCritic). It would also be useful to add a graphical style wherever possible to deliver critiques instead of textual messages. We are also planning a larger evaluation with end-users and to experiment with applying our approach to non-Marama applications.