Comparing Subjective and Objective Data from a Pool of Classical Guitars


M. French[1], L. Lydy[2], and M. Crawford[3]

Abstract - A pool of classical guitars was evaluated subjectively by a group of trained musicians and frequency response functions were measured. The two goals of the effort were to determine whether musicians would largely agree on the tonal quality of the instruments and whether their subjective opinions could be correlated with features in the frequency response functions. These questions are the subject of debate within the community of guitar makers and there is little academic literature that can inform the discussion. The quality of the guitars varied widely, spanning the practical range of instruments available. Subjective results showed that the group tended to agree on the quality of the lowest rated and highest rated instruments. Furthermore, there was no clear correlation between features of measured frequency response functions and subjective ratings.

I.    Introduction

There is a long informal history of making frequency response measurements on acoustic guitars in attempts to link objective measurements, usually frequency response functions (FRFs), with subjective sound quality. However, formal literature on the subject remains rare. There are only a handful of technical books on the physics and design of guitars and academic research is weakly connected to the community of practicing luthiers.

There is a gap in the existing literature on correlating subjective evaluations with measured data. Part of the problem is logistical in that it is very difficult to assemble a collection of guitars along with a group of skilled musicians in controlled circumstances. This article is intended to partially address that gap and was motivated by two questions:

1 Would a group of skilled musicians agree subjectively on the relative desirability of a pool of classical guitars?

2 Is there a clear correlation between subjective evaluations and features in measured frequency response functions?

The lead author has participated in several formal listening experiments and the results have been inconclusive. Collective experience in the guitar making community suggests that players can more readily distinguish between instruments while audiences can more readily distinguish between players. Thus, it is valuable for subjective evaluations to be made by players.

II.    Background

Efforts to correlate subjective sound quality of guitars with various objective measurements have appeared in the literature for some time, starting no later than the early 1980s [1] and several books are in print that describe the basic mechanisms by which acoustic guitars make sound [2] [3] [4]. Many references discuss links between lower resonant frequencies of the body and sound quality, but actionable information is scarce. Accordingly, practice among luthiers is varied. Some tune the structures using subjective or objective targets, while others simply build to a plan, accepting the resulting sound quality. At least one master luthier does not include resonant frequencies at all in a list of the most important aspects of highly successful guitar makers [5].

Efforts at correlation often focus on time domain measures rather than frequency domain ones [6]. Radiation efficiency has also been considered as a means of correlating with subjectively rated sound quality [7]. However, radiation efficiency has the disadvantage that it is difficult to measure, making it unattractive to use as a practical tool for luthiers. Conversely, a clear correlation between sound quality and frequency response functions (FRFs) would be quite valuable since FRFs are relatively easy to measure.

Some published models seek to capture low frequency behavior by matching the lower resonant frequencies. Simple analytical models have been used to explain the basic physics of acoustic guitars since at least 1980, when Christensen and Vistisen proposed a discrete model that captured the lowest two or three resonant frequencies [8]. The model was later refined to include a fourth mode [9]. Methods have been proposed to identify the unknown parameters in these models using measured data [10] and discrete models have been used to predict the effect of structural modifications [11]. These models and the work associated with identifying the necessary parameters are based on resonant frequencies. In this sense, they are useful in understanding how guitars work, but do not describe sound quality. A few articles also explore the value of mathematical descriptions that include approximate mode shapes [12].

The absence of academic literature on the subject has not stopped the community of luthiers from experimenting with the effect of changes in the lower resonant frequencies. Indeed, there is a community of luthiers who tap tune guitar tops, either before they have been incorporated into the guitars, or, less often, after the guitar is partially assembled. Some also modify completed instruments, including production instruments, by altering the structure of the tops. There are some in-depth articles on tuning tops before they have been glued on [13] and some successful luthiers include this free plate tuning (sometimes called voicing) as a part of their building process [14]. These builders sometimes tune to specific frequencies, but often use subjective evaluations during their process. Perhaps the most well-defined approach to tuning the complete instrument is presented by Gore and Gilet [15]. They are also unique in proposing a method to add mass to the instrument to modify mode shapes.

Builders who tune entire instruments after the structure is complete sometimes do target specific frequencies for the lower modes. For example, Gore and Gilet suggest values for the first two resonant frequencies of both steel string and classical guitars. The lowest body frequency of acoustic guitars is strongly conditioned by the enclosed air mass and builders often have a target frequency between 90 Hz and 110 Hz, though there is no universal agreement on preferred frequencies. Individual builders generally build to suit their own tastes or those of their customers.

The state of practice is that there are numerous methods in use that are intended to improve the tonal quality of acoustic guitars by changing the structure to tune the lower resonant frequencies to desired values. However, there is no universal agreement among builders on what those frequencies should be. Furthermore, there appears to be little in the academic literature to support a correlation between subjective sound quality and measured frequencies.

It has been collective wisdom since the 1800s that the materials used for the backs and sides has very little effect on the tonal quality of the instrument. The famous example was an instrument made by Antonio de Torres in 1862 in which the back and sides were made from cardboard [16]. His point was that the back and sides could be made of nearly anything. A modern recording made with this guitar shows that it sounds very good. A recent study more rigorously supports the conclusion the back and side materials are essentially uncorrelated with sound quality [17]. The Leonardo Guitar Research project (sites.google.com/site/leonardoguitarresearch/home) has studied the problem and reached largely the same conclusion. It is worth noting that they also concluded that sound perception is strongly influenced by visually transmitted information. This supports the old and frequently repeated observation that guitar players hear first with their eyes.

III.    Subjective Data Collection

In order for the results to be useful, it was important to test instruments of varying quality and for them to be evaluated by a group of trained musicians. We assembled a group of 12 classical guitars ranging from an inexpensive beginners model to several handmade instruments from skilled luthiers. The musicians were students in the classical guitar program at Purdue University. We focused on classical guitars because of the relative uniformity of their design and because the instruction is formalized. Classical guitar instruction focuses on a specific and refined playing technique that is practiced by all the students in our pool. Our goal was to limit variation in both the design of the instruments and in the styles of the players.

 

The students were asked to evaluate the instruments by playing them and then ranking them in order from most desirable to least desirable. They were given freedom to play whatever pieces they wished. They were also instructed to, as much as possible, to ignore the appearance of the guitar. They were told that the device recording the frequency response functions didnt know what the guitars looked like, so they should avoid letting appearance of the instruments affect their subjective judgement.

 

Subjective evaluations on stringed instruments often cover the players eyes with dark glasses to limit the effect that the appearance of the instruments might have on the results [18]. We elected not to do this because of time limits. A more comprehensive test should prevent the players from clearly seeing the instruments.

 

The testing was conducted in a conference room with low background noise and enough absorptive treatments to eliminate undesirable acoustic reflections. Student evaluators were asked to play in the same chair, with a footrest if they wished, and music stand. They were given 30-40 minutes to evaluate all the guitars and place them in rank order from most to least desirable. The guitars were simply given an identifying number on the headstock. The raw data appears in the Appendix. It is worth noting that the instruments in the pool included some belonging to the students themselves. They were asked to place an asterisk next to their own guitars on their ranking lists. All guitars had fresh strings.

IV.    FRF Measurements

Text Box:  

Photo 1  Frequency Response Testing Setup

We collected driving point frequency response functions from each instrument, using a point on the right side of the bridge. The input was from a modal hammer with a soft tip. The soft tip was selected since we were only looking at the first few modes of the instruments and also because we wanted to avoid any damage to the instruments. In fact, in order to get access to the most expensive instruments, the authors had to assure the owners that we would use the softest available tip for the instrumented hammer. This concentrated impact energy at very low frequencies. As a result, coherence of measured FRFs deteriorated quickly above approximately 300 Hz.

 

The guitars were tuned and placed on soft polyurethane foam blocks to mechanically isolate them from the table on which they rested. The blocks were placed under the sides of the instruments to prevent them from affecting the motion of the backs. The strings were damped using energy absorbing foam earplugs so that only structural response was measured. Photo 1 shows the arrangement.

 

Response was averaged over four taps and Δf=1 Hz. The tapping point was adjacent to the accelerometer mount location, as shown in Photo 1. We had to be very careful as all but one of the instruments were lent for the testing and some were very valuable. The rough treatment sometimes accorded to test instruments in a traditional lab environment was not acceptable in here.

V.    Subjective Results

Since the subjective data is simply the order of the guitars from most to least desirable, the data is presented in histogram form. Since the total number of evaluations is small, the data is presented in thirds (tertiles). Figure 1 shows a histogram of the results, sorted by order in the top tertile. Note that there are several ties according to the top tertile; these are resolved by considering the lower tertiles as well.

Text Box:  

Figure 1  Subjective Choices by Tertile
Guitar 1 was the most expensive one in the test pool and also got the most top tertile choices. It is worth noting, though, that it also got one bottom tertile choice. Conversely, the lowest rated guitar the least expensive in the test pool got only bottom tertile choices. Figure 1 suggests that the players largely agreed on the quality of the most and least desirable guitars. However, the trend was only a general one. Guitar 2 was the most controversial in the group since it got an equal number of top and bottom tertile choices with only one middle tertile choice.

VI.    Objective Results

To quantify the results from the different players, we performed a Friedman test [19] on the ordinal data (Table A2), excluding the results from the second player, who didnt rate instrument 9. The result was p=0.55, which suggests a 55% likelihood that the results could be explained by chance. This shows that there is no statistical consistency in the expert opinions contained in this data set. However, we must acknowledge that modifying the test procedure to cover the eyes of the players might affect the results enough to change the p value.

 

Mode 1 Top and Back Out of Phase

 

Mode 2 Top and Back in Phase

 
Text Box:  

Figure 2  First Body Modes of a Classical Guitar
Access to this pool of guitars, some very expensive and many the primary professional tool of their owners, was limited, so testing had to be done on site and quickly enough to not interrupt subjective evaluations. We also wanted to ensure that environmental conditions did not change between the subjective and objective evaluations. A simple tap test with an instrumented hammer and a very small accelerometer satisfied these constraints.

 

For full sized classical guitars, the first resonant frequency of the instrument is a beam-like bending mode and is typically in the range of 65 Hz 85 Hz. The lower body modes are the first ones that radiate sound. The first body mode is generally around 100 Hz and the second body mode is generally around 200 Hz. Figure 2 shows a graphic of the first two body modes. The third body mode is typically the first one with an internal node line. Both the location of the node line and the frequency vary between guitars, so it is difficult to make a general description of the higher modes.

 

Text Box:  

Figure 3  Waterfall of Measured Response Functions
Part of the purpose of this experiment was to determine whether some feature in the frequency response functions could be correlated with subjective sound quality evaluations. Figure 3 shows a waterfall plot of the measured FRFs, arranged in order of the top tertile, as in Figure 1. There is no clear pattern to the curves and nothing to suggest a simple metric based on the first two frequencies of the instruments correlates with subjective sound quality.

 

Figure 4 shows the same data viewed from the top as a surface plot. Bright line segments represent peaks in the FRFs. Again, there is no obvious pattern to the peaks when arranged in order of top tertile of subjective rating.

 

It is possible that there is some less obvious relationship between subjective sound quality and lower resonant frequencies. However, this data does not support the hypothesis that some combination of target frequencies is correlated with high subjective sound quality.

 

Text Box:  

Figure 4  Surface Plot of Measured Response Functions
Some builders work to ensure that the frequencies of the first two body modes are not exactly an octave apart, that is F3/F2 should not be 2. Table 1 shows resonant frequencies and frequency ratios of the pool of instruments. Rows are shaded by tertile.

 

Two instruments in the test pool have frequency ratios of 2.0 and one of them (Instrument 2) was rated by two players as their top pick. However, One player rated it as the lowest pick and two rated it as second lowest. By this measure, it was the most controversial instrument in the pool. The other instrument with a frequency ratio of 2 (instrument 6) was ranked 10th out of 12 and had no ratings in the top tertile. This data set suggests that F3/F2=2 is not correlated with subjective quality.

 

A note on the authors expectations - in the lead up to this test, we consulted several expert guitar designers and builders, asking whether they thought a group of skilled musicians would largely agree on the rank order of the instruments. The sense of the group was that the evaluators would probably agree, at least roughly. That is, they may disagree on fine distinctions, but the variations in their opinions would be low. Indeed, one of us (French) agreed that this was the expected outcome of the subjective testing. This data set supports that prediction in only the most partial way. There was rough consensus on the most and least desirable instruments, but no clear subjective result beyond that.

 

Text Box: Inst. No.	F1 Beam Bending (HZ)	F2 First Body (Hz)	F3 Second Body (Hz)	F3/F2
1	88	102	196	1.92
8	79	107	202	1.89
4	69	92	191	2.07
2	79	101	202	2
3	76	106	207	1.92
5	74	106	201	1.89
12	69	96	217	2.26
7	83	97	199	2.05
11	72	92	194	2.1
6	75	99	198	2
9	73	104	203	1.95
10	73	99	205	2.07

Table 1  Measured Lower Resonant Frequencies
Finally, we should note that all the players in the group were young. We did not ask their ages, but all were undergraduate students and it is likely that all were younger than 30. It is possible that age affects subjective preferences. Indeed, one expert designer suggested that customers preferences seem to change with age [20]. The authors believe that a test specifically to evaluate the effect of age on subjective preferences might be a valuable addition to the literature.

VII.    Conclusions

A pool of 12 classical guitars of widely varying quality was subjectively evaluated by a group of 7 students in the classical guitar program at Purdue University. The guitars were placed in rank order based on subjective evaluations of the desirability of the instruments. Frequency response functions, measured at the right side of the bridge, were measured from the same instruments. The group agreed that the most expensive guitar, a fine handmade instrument, was the most desirable instrument based on their playing evaluations. They also agreed that the least expensive guitar, an entry level student instrument, was the least desirable instrument in the pool. However, disagreements were sometimes significant and one instrument (No. 2) even got both a highest and a lowest rating from the pool. The players did not agree in a meaningful way on the rank order of the entire pool of instruments.

 

Furthermore, we could not identify a correlation between mean subjective evaluation of the instruments and any feature observable in the frequency response functions. This data set does not support the hypothesis that tuning the lower resonant frequencies of classical guitars to specific values results in superior perceived quality. More specifically, it does not support the hypothesis that F3/F2≈2 results in inferior perceived quality.

VIII.    Text Box: Players =>	1	2	3	4	5	6	7
Most	1	7	4	4	2	2	12
 Desirable	5	1	1	1	8	4	1
 	8	12	3	8	4	1	2
 	3	5	8	11	5	8	3
 	11	4	2	5	11	5	4
 	9	6	7	7	6	11	8
 	12	11	5	12	12	3	11
 	6	8	6	3	3	9	7
 	4	10	11	6	7	12	6
 	10	3	9	10	9	6	5
 Least	2	2	12	9	1	7	10
Desirable	7	 	10	2	10	10	9

Table A1  Rank Order Data from Subjective Evaluations
Appendix

Table A1 Shows the raw subjective point assignments for the test instruments. Shaded cells indicate reviewers own instruments.

 

IX.    Acknowledgements

The authors wish to thank the owners of these guitars for making them available for testing and the students for their conscientious efforts in ranking them. The authors also wish to thank R.M. Mottola for valuable conversations on the role of structural tuning among practicing luthiers and Tim Shaw for his insights on the expectations of players.

 

X.    Bibliography

 

Text Box: Inst No.	1	2	3	4	5	6	7	Comments
1	1	2	2	2	11	3	2	Most expensive instrument in pool.  Handmade, laminated sides, sound ports in upper bout, spruce top
2	11	11	5	12	1	1	3	Handmade, cedar top
3	4	10	3	8	8	7	4	Handmade, spruce top, laminated sides
French polished heel and bracing inside
4	9	5	1	1	3	2	5	Handmade, Nomex double top.  Cedar top ply and spruce inner ply
5	2	4	7	5	4	5	10	Factory made, elevated neck, cedar top
parts on either side of heel
6	8	6	8	9	6	10	9	Factory made, cedar top
7	12	1	6	6	9	11	8	Handmade, spruce top
8	3	8	4	3	2	4	6	Handmade, spruce top, ports on each side of heel
9	6	 	10	11	10	8	12	Factory made, cedar top, approx. $1,800
10	10	9	12	10	12	12	11	Factory made, plywood back and sides, cedar top, approx. $500
11	5	7	9	4	5	6	7	Factory made, spruce top, elevated neck, approx. $2,400
12	7	3	11	7	7	9	1	Factory made, spruce top, laminated sides, approx. $4,200

Table A2  Instrument Rankings and Descriptions
1. J. Meyer, "The Function of the Guitar Body and Its Dependence on Constructional Details," in Function, Construction and Quality of the Guitar, Stockholm, 1983.

2. N. H. Fletcher and T. D. Rossing, The Physics of Musical Instruments, 2nd ed., Springer, 2008.

3. T. D. Rossing, The Science of String Instruments, Springer, 2010.

4. R. M. French, Engineering the Guitar: Theory and Practice, Springer, 2008.

5. R. Brune, "Eight Concerns of Highly Successful Luthiers," American Lutherie, vol. 79, pp. 6-21, 2004.

6. F. Orduna-Bustamante, "Experiments on the Relation Between Acoustical Propoerties and the Subjective Quality of Classical Guitars," CAS Journal, vol. 2, no. 1, pp. 20-23, 1992.

7. R. Boullosa, F. Orduna-Bustamante and A. Perez Lopez, "Tuning Characteristics, Radation Efficiency and Subjective Quality of a Set of Classical Guitars," Applied Acoustics, vol. 56, no. 3, pp. 183-197, 1999.

8. O. Christensen and B. Vistisen, "Simple Model for Low Frequency Guitar Function," Journal of the Acoustical Society of, vol. 68, no. 3, pp. 758-766, 1980.

9. J. Popp, "Four Mass Coupled Oscillator Guitar Model," Journal of the Acoustical Society of America, vol. 131, no. 1, pp. 829-836, 2012.

10. T. Hill, B. Richardson and S. Richardson, "Acoustical Parameters for the Characterisation of the Classical Guitar," Acta Acoustica United with Acustica, vol. 4, pp. 335-348, 2004.

11. R. M. French, "Structural Modification of Stringed Instruments," Mechanical Systems and Signal Processing, vol. 21, no. 1, pp. 98-107, 2007.

12. B. E. Richardson, "Simple Models as the Basis for Guitar Design," CAS Journal, vol. 4, no. 5, pp. 30-36, 2002.

13. Carruth, "Free Plate Tuning," in Big Red Book of American Lutherie, Vol 3, Guild of American Luthiers, 2004, pp. 136-172.

14. D. Bourgeois, "YouTube - Dana Bourgeois Guitar Voicing," Bourgeois Guitars and Northern Lights Music, 25 Sept 2017. [Online]. Available: https://youtu.be/Ei5-DkVTrEE. [Accessed 7 May 2019].

15. T. Gore and G. Gilet, Contemporary Acoustic Guitar Design and Build, Trevor Gore, 2011.

16. S. Grodona, L. Waldner and M. Mandelli, Masterieces of Guitar Making, L'officina del Libro, 2001.

17. S. Carcagno, R. Bucknall, J. Woodhouse, C. Fritz and C. Plack, "Effect of Back Wood Choice on the Perceived Quality of Steel-String Acoustic Guitars," Journal of the Acoustical Society of America, vol. 144, no. 6, pp. 3533-3547, 2018.

18. Saitis, C. Fritz, G. Scavone, C. Gustavino and D. Dubois, "Perceptual Evaluation of Violins: A Psycholinguistic Analysis of Preference Verbal Descriptions by Experienced Musicians," Journal of the Acoustical Society of America, vol. 141, no. 4, pp. 2746-2757, 2017.

19. E.B. Page, Ordered Hypotheses for Multiple Treatments: A Significance Test for Linear Ranks, Journal of the American Statistical Association, vol. 58, pp. 216-230, 1963.

20. T. Shaw, Interviewee, Conversation on Customers' Expectations. [Interview]. May 2019.



[1] Mark French, Professor, School of Engineering Technology, Purdue University, 121 Knoy Hall, 401 N. Grant St., West Lafayette IN 47907

[2] Laura Lydy, Director of Guitar/String Studies, School of Music, Purdue Fort Wayne, 21010 E. Coliseum Dr., Ft. Wayne IN, 46805

[3] Miles Crawford, Graduate, School of Engineering Technology

 

Manuscript received June, 2019.