DYNA

1. Introduction

To generate new solutions to a product design problem it is usual to apply creativity methods that make it easy to explore and generate new solutions. The importance of creativity has been highlighted in many studies, and universities must promote good practices that foster these capabilities in order to create complete professionals [1]. Moreover, it is important to enhance it from childhood onwards [2].

The idea-generation process is essential to obtain creative design solutions. Thus, there are several approaches to evaluate the effectiveness of the ideation process, such as that of Shah [3], who proposed four metrics: the novelty, variety, quality, and quantity of ideas generated. Here, quantity is defined as the degree to which an idea fulfils the design problem requirements.

The quality of the design solutions is related to different aspects, such as feasibility and effectiveness, and it is also linked to how productive the design process is. In the monograph edited by Duffy [4], the elements to measure design process productivity comprise the extent to which a solution fulfils the design problem requirements and needs, the cost, the time or the technical requirements.

In this study, feasibility is considered to be the capability to implement a solution in practice, considering technological and cost criteria. Effectiveness is understood as the way in which the design solution provides an answer to the problem. Moreover, the term 'creativity' has been defined in different ways [5], all of them sharing the common notion that something is creative when it is novel and useful [6].

In a previous experiment, the degree of creativity of several design solutions obtained by applying different design methods was measured and compared. In this experiment, twelve designers were distributed in four teams of three members, one of them being a designer, another an engineer and the third one could be either a designer or an engineer. Four design problems were defined: a drawing table which occupied as little space as possible (P1), a tubular map case allowing for one-by-one extraction and introduction of maps (P2), a system to hide leads in tables (P3), and a table that allowed the user to work either seated or standing up (P4).

A criterion that is frequently used to classify design methods belongs to the intuitive methods, which allow the spontaneous generation of ideas and logical or structured methods, which in turn lead creative thinking to follow several steps in a more defined path. One of the best-known intuitive methods is brainstorming and all its variants. This method consists in the spontaneous generation of as many ideas as possible in a team, avoiding any kind of judgement or criticism [7]. Among the structured methods, functional analysis [8] analyses the functions in a new design and their relationships in order to search for potential solutions. The SCAMPER method is a lateral thinking technique that promotes the generation of ideas based on a list of questions that ask, among other things, which design elements can be substituted, adapted or used in a different way [9].

This experiment was organised in such a way that each team had to solve one of the design problems without applying any prescribed method (no method) and then they solved two other design problems by applying different design methods, namely two types of brainstorming (BR1) and (BR2), and the SCAMPER and functional analysis methods, as shown in Table 1. As can be seen, each of the four methods (BR1, BR2, functional analysis and Scamper) were applied by two different teams in two different design problems during the experiment.

Table 1: Design experiment

Source: Own elaboration

Once the experiment had ended, the degree of creativity was measured by means of the degree of novelty and usefulness of the solutions in two different ways: by applying the SAPPhIRE model, and through experts’ opinion. The SAPPhIRE model, which stands for State-Action-Part-Phenomenon-Input-oRgan-Effect [10], is used to assess the relative degree of novelty depending on the level on which the change is identified in the causality model that describes the product functionality at different levels of abstraction. Usefulness is obtained by means of a mathematical formula using the importance of the product, the usage frequency, the duration of the benefit and the popularity ratio as input variables. With this measuring method, the different solutions obtained for a design problem were classified according to their novelty, usefulness and creativity. The most novel and useful solution was scored with a 1, the second with a 2 and so on. Lastly, creativity was measured as the product between the other two metrics: novelty and usefulness [11,12].

The degree of creativity of the results of the experiment, measured with Sarkar and Chakrabarti’s method, showed that brainstorming stimulates more novel solutions and that the method that generates more useful solutions is functional analysis, which can probably be due to the fact that this method leads human thinking towards the functionality of the solution [13].

One of the disadvantages of evaluating a solution after a creative process, and more so if the study is conducted in an academic context, is the difficulty involved in valuing the technical and economic feasibility of the solution, since this is defined at a very conceptual level and there is no specific information about the manufacturing technologies and materials.

Thus, the quality of the conceptual solutions generated in the experiment were evaluated by means of an opinion questionnaire answered by eight external experts with professional experience in the design of furniture or similar products. Nonetheless, only seven of them were considered for the analysis, since one of them presented a deviation higher than 3 sigma. The results show that experts also rated the solutions generated when functional analysis was applied as the most feasible. Solutions generated with SCAMPER and no method were less feasible, and even less so with brainstorming. When no design method was prescribed, the feasibility was lower than with functional analysis [14]. Experts also rated the solutions generated using functional analysis as more effective, followed by those obtained when SCAMPER, no method and brainstorming were applied (Fig. 1). The differences were lower than for feasibility, that is, the design method had a lower influence on effectiveness than on feasibility. Again, functional analysis was the best one, probably because it enables the designer to focus his/her mind on searching for design solutions that solve the design problem.

Several studies have carried out experimental analyses of creative stimuli and methods in the effectiveness of the creative design process [15], [16] and [17]. For example, in López-Mesa et al. [18], the feasibility of the ideas generated was studied under the hypothesis that the more time devoted to developing an idea during the design process, the higher feasibility will be. This analysis was performed by comparing the time devoted to ideas when visual stimuli were used and when the SCAMPER questions were used as stimuli. The results showed that SCAMPER led to the generation of more feasible ideas than when images were used as external stimuli in a brainstorming method.

Figure 1: Mean feasibility and effectiveness in the design solutions for the different design methods

Source: Own elaboration

The appropriateness of brainstorming as a method to obtain more novel ideas has also been analysed in a study [19] which compared the design results obtained using TRIZ and those obtained using SCAMPER, it was observed that the design solution generated by a team that used brainstorming showed the most novel value, but its usefulness was negative.

These analyses provide information that helps in the selection of the most appropriate method during the creative phase of the design process. Considering that obtaining very creative ideas is linked with ideas that are difficult to implement in practice, but, at the same time, it is very important to promote creativity within the innovation process, it would be interesting to analyse the possible relation between the degree of novelty of the design solutions and their quality.

Is it true that the more creative ideas are less feasible and less effective? And, conversely, when the solutions generated are less creative, are they more feasible and better fulfil the objectives and requirements of the problem? The objective of this study is to analyse the relationship between feasibility and effectiveness and the degree of novelty, usefulness and creativity of the design outcomes.

2. Methodology

The design outcomes generated during the experiment are depicted in Table 2, and consist of three different solutions for each design problem, each one generated by a different design team and using a different design method. All the teams were asked to draw and describe the final solution decided on by the team using pencils and paper, although one team built up a paper mock-up for one of the solutions. The first row in Table 2 shows the results for the drawing table (P1), the second one the outcomes for the tubular map case (P2), the third row contains the solutions of three design teams for the system to hide leads in tables (P3) and the last row shows the design outcomes for the table to work seated and standing up (P4). Therefore, there are three different design outcomes for each of the design problems proposed.

Table 3 shows the questionnaire answers about the degree of feasibility and effectiveness according to the experts’ opinion [14], using a scale of 1 to 10 (where 10 is the highest and 1 the lowest). The values for the degree of novelty, usefulness and creativity employed in this study have been obtained in two ways, as described in detail in [13], in which a normalised scale between 0 and 1 was employed. In Table 3, a scale of 1 to 3, instead of a normalised one, was used, where 1 is the most novel, useful and creative and 3 the least. The two ways to assess it are:

- The experts’ criteria, by means of a questionnaire.

- Creativity method defined by Sarkar and Chakrabarti [12].

Table 2: Design outcomes obtained by each design team, for the four design problems using different design methods

Source: Reference [13]

Table 3: Mean feasibility and effectiveness, and order of novelty, usefulness and creativity according to experts and to Sarkar and Chakrabarti.

Source: Own elaboration.

To study the potential relation between feasibility and effectiveness with the parameters that define creativity (novelty and usefulness), scatter diagrams were plotted for each of the pairs to be compared: feasibility-novelty, effectiveness-novelty, feasibility-usefulness, effectiveness-usefulness, feasibility-creativity and effectiveness-creativity. Several regression models were applied to the scatter diagrams in order to identify potential relationships, and the regression curves for the data are depicted.

3. Results

The following figure represents each of the design outcomes obtained in the experiment in such a way that the position on the y-axis is the order in the degree of novelty and its x-axis position is the feasibility score (Fig. 2a) and the effectiveness score (Fig. 2b). Both the degree of novelty according to the experts’ opinion and to the SAPPhIRE method are represented.

No significant fit is found, since none of the R2 values are high enough to consider the fit as significant. In any case, a tendency is observed in some cases. As can be seen, novelty, as measured by the experts, decreases when the feasibility increases, that is, the more novel the solution is, the less feasible it seems to be. The curve in Fig. 2a represents the potential regression tendency of the data. When novelty is assessed with the SAPPhIRE method, there is one outcome with high feasibility that is also very novel, and, consequently, even though there also seems to be a relationship, it is less clear than when novelty was evaluated by the experts.

This could be due to the fact that the most novel outcomes usually involve complex or novel technologies, which are still under development or it is the result of not taking into account all the needs of the problem. This is what has happened, for instance, with the virtual table that uses holographic systems.

Fig. 2b shows that there is no relation between the degree of effectiveness and novelty, since there are very novel solutions that are very effective and very novel solutions that have low effectiveness, in both the experts’ criteria and the SAPPhIRE method. That is, the designers generate design solutions whose degree of novelty appears to be unrelated to how well the solution solves the design problem.

Figure 2: Relationship between the degree of novelty and a) feasibility, and b) effectiveness

Source: Own elaboration

Regarding the degree of utility of the solution, Fig. 3a shows that there is no relation between this and the degree of feasibility, since the dots that represent the design outcome scores on usefulness and feasibility are highly dispersed in the scatter plot. In other words, during the creative ideation of ideas, when designers still do not have to think about technical details, the design process does not lead to a different level of feasibility depending on the usefulness parameters, which take into account the duration and frequency of usage.

Figure 3: Relationship between usefulness and a) feasibility, and b) effectiveness

Source: Own elaboration

Fig. 3b shows a possible relationship between effectiveness and usefulness in such a way that the more effective the design outcome is, the more useful it is in both the experts’ opinion and the SAPPhIRE method. Hence, when the design method produces more effective outcomes, they are also more useful. This relationship is stronger when the usefulness is evaluated by the experts, maybe because in their own criteria, usefulness is very similar to effectiveness.

The relationship between usefulness and effectiveness might not be so direct in other kinds of design problems to those analysed in this study. One example of this could be if the design requirements do not consider generating solutions that improve the usage of the products, as could happen in a design problem concerning an anecdotal need.

Lastly, the scatter plots of the degree of creativity are depicted in Fig. 4. The first observation is that the solutions obtained present more similarities in the degree of creativity than in novelty and usefulness, since all the dots are closer to each other on the y-axis. Moreover, there is no relationship between creativity and feasibility or between creativity and effectiveness. Since creativity comprises both novelty and usefulness [5], the reason why there not many differences may be that there are very novel solutions with a low degree of usefulness and vice versa. It is more difficult to generate solutions with high novelty and at the same time high usefulness, and that is why creativity values are lower.

Figure 4: Relationship between the degree of creativity and a) feasibility, and b) effectiveness

Source: Own elaboration

So, assuming that to generate creative solutions they need to be both novel and useful, it has to be taken into account that when the design outcomes are more novel, the feasibility tends to be lower. If design methods like brainstorming and SCAMPER are used to generate creative solutions, it is advisable to conduct a feasibility analysis and, if needed, research new ideas to increase the feasibility. These methods usually lead to highly creative ideas [13] and so they are appropriate even though they can lead to low feasibility.

In future research it could be interesting to analyse the extent to which the expressive ability to draw and describe the design outcomes might have affected the evaluators' perception and, consequently, their final scores.

4. Conclusions

This study analyses the relationship between feasibility and effectiveness and the values for novelty, usefulness and creativity. The results show that the higher the degree of novelty is, the lower the feasibility of the design outcomes will be. Hence, when designers are generating creative ideas, it is recommended that a review of the feasibility should be conducted in those design outcomes that present a high level of novelty. Failing that, the design objectives should be redefined, since it is also common that creative ideas imply complex or emergent technologies, which are difficult to implement in practice. This is what has happened in design outcomes like the one in which a holographic system was proposed.

Moreover, there seems to be a relation between the degree of effectiveness of the design outcome and the degree of usefulness. Consequently, a good level of resolution of the design problem seems to be related to solutions that can be used more, at least in the design problems employed in this study. This is the case of the outcomes generated with functional analysis, which present a higher score on usefulness and also on effectiveness.

Thus, since when the feasibility increases, novelty is lower and usefulness remains similar, creativity, as the combination of novelty and usefulness, should decrease. Looking at Fig. 4a it can be seen that, with the collected data, this relationship has a very low correlation and consequently it cannot be assessed, although there is a gentle trend to decrease creativity when feasibility increases. This trend is stronger in the experts’ evaluation than in the SAPPhIRE method.

Likewise, since when effectiveness increases, usefulness is higher and novelty remains the same, creativity should increase. However, with the data collected in this experiment, no relationship can be determined, since the data are very dispersed (Fig. 4b).

These conclusions agree with previous studies, such as Chulvi et al. [19], in which brainstorming led to higher novelty, but usefulness was low. The present study has allowed us to confirm this pattern in a larger number of cases, since the design outcomes are more novel but less useful when the brainstorming method is applied.