Opinion or fact?

 Cognitive Film Theory, Film Analysis, Film Studies, Film Style, Silent cinema  Comments Off
Dec 152011
 

The Artist has been wowing audiences across the world. The film has already won some awards, and is hotly tipped for many more. It has also been attracting much interest in the press, and film scholars have been roped into this.

In an interview with the BBC, silent film expert Briony Dixon of the BFI made a series of statements that are worth reflecting upon:

  1. watching silent films is more rewarding than watching contemporary Hollywood action blockbusters
  2. watching a silent film requires more work on the part of the viewer
  3. slower edited films require greater concentration than rapidly edited films

You can view the video of the interview here. The text on this web page includes the following sentence:

Bryony Dixon, a silent film expert from the BFI, told BBC News that because silent films require more concentration, the rewards of watching them are richer than action blockbusters.

So let’s take these three statements in turn:

1. Watching silent films are more rewarding that watching contemporary Hollywood films

I am aware of no research that compares the viewing pleasures derived from silent films to sound films, and I have not been able to find any such research. In fact, what viewers find rewarding about the film experience is an under-researched area of film studies. If anyone knows of any research in this area please feel free to add a comment to this post listing the appropriate references.

This is just Dixon’s opinion, and we should not be surprised that an expert on silent films should prefer silent films. Other people will have their own opinions, tastes, and preferences. The difference is that other people will not have the opportunity to express them in the BBC under the heading ‘Expert on the rewards of silent film.’ This is problematic because it presents Dixon’s opinion as fact (‘An expert says …’). This may be the fault of the BBC and the way it has presented the interview, but from watching the video I doubt it.

Of course, a factor here  is that there has not been much in the way of silent film since 1930 and so research on what viewers think about silent films has inevitably been extremely limited. The Artist provides an excellent opportunity for researchers to engage with this topic.

2.Watching a silent film requires the viewer to work harder

There is no research that I can find looking at the cognitive load of silent cinema (probably for reasons noted above), and the literature on cognitive load in film viewing is somewhat limited in general.  An interesting place to start is this paper from Nitzan Ben-Shaul:

Ben-Shaul N 2003 Split attention problems in interactive moving audiovisual texts, Fifth International Digital Arts and Culture Conference, Melbourne, Australia, 19-23 May, 2003.

It is also worth reading Julian Hochberg and Virginia Brooks’s work on film viewing and visual momentum as it gives a general description of how observers attend to images (both moving and still) and how we cognitively process this information:

Hochberg J and Brooks V 1978 Film cutting and visual momentum, in JW Senders, DF Fisher, and RA Monty (eds.) Eye-movements and the Higher Psychological Functions. Hillsdale, NJ: Erlbaum: 293-313.

Hochberg J and Brooks V 1996 Movies in the mind’s eye, in D Bordwell and N Carroll (eds.) Post-Theory: Reconstructing Film Studies. Madison, WI: UNiversity of Wisconsin Press: 368-387.

Cognitive load theory (CLT) might support the opposite conclusion to Dixon’s assertion. According to CLT, we have only a limited amount of working memory and the cognitive load of a task is determined by the number and complexity of the steps involved that use up those resources. The following example is from Gutashaw WE and Brigham FJ 2005 Instructional support employing spatial abilities: using complimentary cognitive pathways to support learning in students with achievement deficits, in TE Scruggs, MA Mastropieri (eds.) Cognition and Learning in Diverse Settings: Amsterdam: Elsevier: 47-70.

Watching a film in a language one does not understand but with subtitles is an example of an increased cognitive load over watching the same film in one’s own language. Now image watching a subtitled film with poor reading skills. The cognitive load increases dramatically (66).

Thinking along similar lines, we might think that because we do not have to attend to dialogue as well as images that the cognitive load in watching a silent film is lower than that when watching a film with synchronised dialogue that requires attention to multiple sensory modalities.

There has been no direct research on cognitive load that could answer this question, and so I make this argument as a hypothesis only, but as we see in relation to the next point the evidence indicates it is faster editing that increases the cognitive load on the viewer.

Cognitive load theory does play an important role in the media theory of Richard Mayer and Roxana Moreno, and you can find an introduction their research here: Mayer RE and Moreno R 1998 A cognitive theory of multimedia learning: implications for design principles, ACM SIGCHI Conference on Human Factors in Computing Systems, 18-23 April 1998, Los Angeles.

Dixon’s statement sound plausible, but without supporting research it is nothing more than a hypothesis and there are other hypotheses to be made and tested on this point. Of course, it may be that I just haven’t been looking for research in the right places and so if anyone knows of research demonstrating if this statement is true or not then please let me know.

3. Films edited more slowly require more concentration than rapidly cut films

There are a couple of things to consider here. First, contemporary film audiences are less likely to be familiar with silent films than they are with modern action blockbusters. Therefore, they may concentrate  more on something unfamiliar than something commonplace and this would account for a difference in viewers’ experience. We may find that with increasing experience viewing habits may change so that viewers familiar with both silent films and contemporary cinema watch them in the same way. Again, this relates to the cognitive load placed in the viewer. This sounds plausible, but as noted above I have been able to find no research in this topic. In fact I can find no research on viewers’ ‘concentration’ in the cinema, and this leads us to our second problem: what is meant by ‘concentration?’ Dixon never defines the terms she uses, and it may mean the number of times a viewer looks at the screen, the length of time the viewer looks at a screen, the focus of the viewer’s attention when looking at the screen, etc.

If we take concentration to mean something similar to attention, then there is some research on this topic and it contradicts Dixon’s assertion that slower films require more concentration than fast edited films. Research on the limited capacity model of viewership has shown that rapid pacing in motion pictures requires increased allocation of perceptual resources. The research can be read in this paper:

Lang A, Bolls P, Potter RF, and Kawahara K 1999 The effects of production pacing and arousing content on the information processing of television messages, Journal of Broadcasting and Electronic Media 43 (4): 451-475.

The limited capacity model defines the viewer as an information processor faced with a variably redundant ongoing stream of audio-visual information, in the message content is the topic, genre, and information contained in a message. Therefore, ‘viewing is the continuous allocation of a limited pool of processing resources to the cognitive process required for viewers to make sense of a message.’ (I don’t like this definition of message content – it seems somewhat circular to me).

This research looked at the effect of production pacing and content on attention in the cinema, testing the hypothesis that both pacing and arousing content should increase the level of resources automatically allocated to processing the message. The results showed this is indeed the case: arousing content and fast pace increased self-reported arousal in television viewers, and that both factors increase the allocation of resources to processing messages.

This is also discussed in a subsequent paper (below), which showed that faster pacing resulted in the allocation of greater resources by viewers in attending to a television message and that self-reported arousal also increased with editing pace.

Lang A, Zhou S, Schwartz N, Bolls PD, and Potter RF 2000 The effects of edits on arousal, attention, and memory for television messages: when an edit is an edit can an edit be too much?, Journal of Broadcasting and Electronic Media 44 (1): 94-109.

In summary, there is no evidence that slower films require greater concentration by film viewers but there is evidence that faster paced films – such as (but obviously not limited to) action blockbusters – do elicit greater allocation of information processing resources (including attention).

A final point to make is that we do not yet know what the distribution of shot lengths in The Artist, and so comparing its pace to other films is not yet possible. It will be interesting when the film comes out on DVD and we can look at it frame-by-frame to see whether its editing style is compatible with contemporary cinema or with silent films of the 1920s. However, as yet we cannot make any empirical statement about the contribution of editing to the pace of this film.

Dixon’s comments raise some interesting questions about the nature of film viewing and silent cinema, but in the absence of supporting evidence they are opinions and not facts. The danger comes when we accept the former as the latter without asking questions or referring to the existing research in this area. Empirical research leads us to reject incorrect and empty opinions by establishing the nature of those facts. This is what film studies is supposed to be for.


The Mann-Whitney U Test

 Cinemetrics, Film Analysis, Film History, Film Studies, Film Style, Film Technology, Hollywood, Laurel and Hardy, Silent cinema, statistics  No Responses »
May 122011
 

There is a dire need for film scholars to understand elementary statistics if they intend to use it to analyse film style. See here for the problems a lack of statistical education creates.

This post will illustrate the use of the Mann-Whitney U test using the median shot lengths of silent and sound Laurel and Hardy short films produced between 1928 and 1933 (see here). I will also look at effect sizes for interpreting the result of the test. Before proceeding, it is important to note that the Mann-Whitney U test goes by many different names (Wilcoxon Rank Sum test, Wilcoxon-Mann-Whitney, etc) but that these are all the same test and give the same results (although they may come in a slightly different format).

The Mann-Whitney U test

The Mann-Whitney U test is a nonparametric statistical test to determine if there is a difference between two samples by testing if one sample is stochastically superior to the other (Mann and Whitney 1947). By stochastic ordering we mean that data values from one sample (X) are more likely to assume small values than the data values from another sample (Y) and that the data values in X are less likely to assume high values than Y.  If Fx(z) ≥ Fy(z) for all z, where F is the cumulative distribution function, then X is stochastically smaller than Y.

We want to find out if there is a difference between the median shot lengths of silent and sound films featuring Laurel and Hardy. The null hypothesis for our experiment is that

the two samples are stochastically equal

(Ho: Fsilent (z) = Fsound (z) for all z).

In other words, we assume that there is no difference between the samples – the median shot lengths of the silent films of Laurel and Hardy are no more likely to be greater or less than the median shot lengths of the sound films of Laurel. (See Callaert (1999) on the nonparametric hypotheses for the comparison of two samples).

In order to perform the Mann-Whitney U test we take our two samples – the median shot lengths of the silent and sound films – and we pool them together to form a single, large sample. We then order the data values from smallest to largest and assign a rank to each value. The film with the smallest median shot length has a rank 1.0, the film with second smallest median shot length has a rank of 2.0, and so on. If two or more films have a median shot length with the same value, then we give each film rank an average rank. For example, in Table 1 we see that five films have a median shot length of 3.3 seconds and that these films are 5th, 6th, 7th, 8th, and 9th in the ordered list. Adding together these ranks and dividing by the number of tied films gives us the average rank of each film: (5 + 6 + 7 + 8 + 9)/5 = 7.0.

Table 1 Rank-ordered median shot lengths of Laurel and Hardy silent (n = 12) and sound (n = 20) films

Notice that in Table 1, the silent films (highlighted blue) tend to be at the top of the table with lower rankings than the sound films (highlighted green) that tend to be in the bottom half of the table with the higher rankings. This is a very simple way to visual the stochastic superiority of the sound films in relation to the silent films. If the two samples were stochastically equal then we would see more mixing between the two colours.

Now all we need to do is to calculate the U statistic. First, we add up the ranks of the silent and sound films from Table 1:

Sum of ranks of silent films = R1 = 1.0 + 4.0 + 7.0 + 7.0 + 7.0 + 10.5 + 12.0 + 13.0 + 14.0 + 18.0 +18.0 +22.5 = 134.0

Sum of ranks of sound films = R2 = 2.0 + 3.0 + 7.0 + 7.0 + 10.5 + 18.0 + 18.0 + 18.0 + 18.0 +18.0 +22.5 +24.0 + 25.0 + 26.0 + 27.0 + 28.5 + 28.5 + 30.0 + 31.0 + 32.0 = 394.0

Next, we calculate the U statistics us the formulae:

where n1 and n2 are the size of the two samples, and R1 and R2 are the sum of ranks above. For the above data this gives us

We want the smallest of these two values of U, and the test statistic is, therefore, U = 56.0. (Note that U1 + U2 = n1 × n2 = 240).

To find out if this result is statistically significant we can compare it to a critical value for the two sample sizes: as n1 = 12 and n2 = 20, the critical value when α = 0.05, is 69.0. We reject the null hypothesis if the value of U we have calculated is less than the critical value, and as 56.0 is less than 69.0 we can reject the null hypothesis of stochastic equality in this case and conclude that there is a statistically significant difference between the median shot lengths of the silent films and those of the sound films. As the median shot lengths of the sound films tend to be larger than the median shot lengths of the silent films we can say that they are stochastically superior.

Alternatively, if our sample is large enough then U follows a normal distribution and we can calculate an asymptotic p-value using the following formulae:


For the above data, U = 56.0, μ = 120.0, and σ = 25.69. Therefore z = -2.49, and we can find the p-value from a standard normal distribution. The two-tailed p-value for this experiment is 0.013. (Note that ‘large enough’ is defined differently in different textbooks – some recommend using the z-transformation when both sample sizes are at least 20 whilst others are more generous and recommend that both sample sizes are at least 10).

If some more restrictive conditions are applied to the design of the experiment, then the Mann-Whitney U test is a test of a shift function (Y = X + Δ) for the sample medians and is an alternative to the t-test for the two-sample location problem. Compared to the t-test, the Mann-Whitney U test is slightly less efficient when the samples are large and normally distributed (ARE = 0.95), but may be substantially more efficient if the data is non-normal.

The Mann-Whitney U test should be preferred to the t-test for comparing the median shot lengths of two groups of films even if the samples are normal because the former is a test of stochastic superiority, while the latter is a test of a shift model and this is not an appropriate hypothesis for the design of our experiment. It simply doesn’t make sense to speak of the median shot length of a sound film in terms of a shift function as the median shot length of a silent film plus the impact of sound technology. You cannot take the median shot length of Steamboat Bill, Jr (X), add Δ number of seconds to it, and come up with the median shot length of Dracula (Y = X + Δ). Any such argument would be ridiculous, and only the null hypothesis of stochastic equality is meaningful in this context.

The probability of superiority

A test of statistical significance is only a test of the plausibility of the model represented by the null hypothesis. As such the Mann-Whitney U test cannot tell us how important a result is. In order to interpret the meaning of the above result we need to calculate the effect size.

A simple effect size that can be quickly calculated from the Mann-Whitney U test statistic is the probability of superiority, ρ or PS.

Think of PS in these terms:

You have two buckets – one red and one blue. In the red bucket you have 12 red balls, and on each ball is written the name of a silent Laurel and Hardy film and its median shot length. In the blue bucket you have 20 blue balls, and on each ball is written the name of a sound Laurel and Hardy film and its median shot length. You select at random one red ball and one blue ball and note down which has the larger median shot length. Replacing the balls in their respective buckets, you draw two more balls – one from each bucket – and note down which has the larger median shot length. You repeat this process again, and again, and again.

Eventually, after a large number of repetitions, you will have an estimate of the probability with which a silent films will have a median shot length greater than that of a sound film. (On Bernoulli trials see here).

The probability of superiority can be estimated without going through the above experiment: all we need to do is to divide the U statistic we got from the Mann-Whitney test by the product of the two sample sizes – PS = U/(n1 × n2). This is equal to the probability that the median shot length of a silent film (X) is greater than the median shot length of a sound film (Y) plus half the probability that the median shot length of a silent film is equal to the median shot length of a sound film: PS = Pr[X > Y] + (0.5 × Pr[X = Y]).

If the median shot lengths of all the silent films were greater than the median shot lengths of all the sound films, then the probability of randomly selecting a silent film with a median shot length greater than the median shot length of sound film is 1.0.

Conversely, if the median shot lengths of all the silent films were less than the median shot lengths of all the sound films, then the probability of randomly selecting a silent film with a median shot length greater than the median shot length of sound film is 0.0.

If the two samples overlap one another completely, then the probability of randomly selecting a silent film with a median shot length greater than the median shot length of sound film is equal to the probability of randomly selecting a silent film with a median shot length less than the median shot length of a sound film, and is equal to 0.5.

So if there is no effect PS = 0.5, and the further away PS is from 0.5 the larger the effect we have observed.

There are no hard and fast rules regarding what values of PS are ‘small,’ ‘medium,’ or ‘large.’ These terms need to be interpreted within the context of the experiment.

For the Laurel and Hardy data, we have U = 56.0, n1 = 12, and n2 = 20. Therefore, PS = 56/(12 × 20) = 56/240 = 0.2333.

Let us now compare the effect size for the Laurel and Hardy paper with the effect size from my study on the impact of sound in Hollywood in general (access the paper here). For the Laurel and Hardy data PS = 0.2333, whereas for the Hollywood data PS = 0.0558. In both studies I identified a statistically significant difference in the median shot lengths of silent and sound films, but it is clear that the effect size is larger in the case of the Hollywood films than for the Laurel and Hardy films.

The Hodges-Lehmann estimator

If we have designed our experiment to understand the impact of sound technology on shot lengths in Laurel and Hardy films around a null hypothesis of stochastic equality, then it makes no sense to subtract the sample median of the silent films from the sample median of the sound films because this implies a shift function and therefore a different experimental design and a different null hypothesis.

If we are not going to test for a classical shift model, how can we estimate the impact of sound technology on the cinema in terms of a slowing in the cutting rate?

To answer this question, we turn to the Hodges-Lehmann estimator for two samples (HLΔ), which is the median of the all the possible differences between the values on the two samples.

In Table 2, the median shot length of each of the Laurel and Hardy silent films is subtracted from the median shot length of each of the sound films. This gives us a total set of 240 differences (n1 × n2 = 12 × 20 = 240).

Table 2 Pairwise differences between the median shot lengths of Laurel and Hardy silent films (n = 12) and sound films (n = 20)

If we take the median of these 240 differences we have our estimate of the typical difference between the median shot length of a silent film and the median shot length of a sound film. Therefore, the average difference between the median shot lengths of the silent Laurel and Hardy films and the median shot lengths of the sound Laurel and Hardy films is estimated to be 0.5s (95%: 0.1, 1.1). (I won’t cover the calculation of the (Moses) confidence interval for the estimator HLΔ in this post, but for explanation see here).

The sample median of the silent films is 3.5s and for the sound films it is 3.9s, and the difference between the two is 0.4s, but as the shift function is an inappropriate design for our experiment this actually tells us nothing. Now it would appear that the difference between the two sample medians and HLΔ are approximately equal: 0.4s and 0.5s, respectively. But it is important to remember that they represent different things and have different interpretations. The difference between the sample medians represents a shift function, whereas the Hodges-Lehmann estimator is the average difference between the median shot lengths.

Note than we can calculate the Mann-Whitney U test statistic directly from the above table. If we count the number of times a silent film has a median shot length greater than that of a sound film (i.e Δ < 0, the green-highlighted numbers) and add this to half the number of times the silent and sound films have equal median shot lengths (i.e. Δ = 0, the red-highlighted numbers), then we have the Mann-Whitney U statistic that we derived above: U2 = 47 + (0.5 × 18) = 56. Equally, if we add the number of times a silent film has a median shot length less than that of sound film (i.e. Δ > 0, the blue-highlighted numbers) to half the number of times the medians are equal, then we have U1 = 175 + (0.5 × 18) = 184.

Bringing it all together

Once we have performed out hypothesis test, calculated the effect size, and estimated the effect we can present our results:

The median shot lengths of silent (n = 12, median = 3.5s [95% CI: 3.2, 3.7]) and sound (n = 20, median  = 3.9s [95% CI: 3.5, 4.3]) short films featuring Laurel and Hardy produced between 1927 and 1933 were compared using a Mann-Whitney U test, with a null hypothesis of stochastic equality. The results show that there is a statistically significant but small difference of HLΔ = 0.5s (95% CI: 0.1, 1.1) between the two samples (U = 56.0, p = 0.013, PS = 0.2333).

These two sentences provide a great deal of information to the reader in a simple and economical format – we have the experimental design, the result of the test, and the practical significance of the result.

Note that at no point in conducting this test have we employed a ‘dazzling array’ of mathematical operations – in fact the most complicated thing in the while process was to find the square route in the equation for σ above and everything else was numbering items in a list, addition, subtraction, multiplication, or division.

Summary

The Mann-Whitney U test is ideally suited to our needs in comparing the impact of sound technology on film style, and has numerous advantages over the alternative statistical methods:

  • it is covered in pretty much every statistics textbook you are ever likely to read
  • it is a standard feature in statistical software (though you will have to check which name is used) and so you won’t even have to do the basic maths described above
  • it is easy to calculate
  • it is easy to interpret
  • it allows us to test for stochastic superiority rather than a shift model
  • it is robust against outliers
  • it does not depend on the distribution of the data
  • it can be used to determine an effect size (PS) that is easy to calculate and simple to understand
  • we have a simple estimate of the effect (HLΔ) that is consistent with the test statistic

If you want to compare more than two groups of films, then the non-parametric k-sample test is the Kruskal-Wallis ANOVA test (see here). The Mann-Whitney U test can also be applied as post-hoc test for pairwise comparisons.

References and Links

Callaert H 1999 Nonparametric hypotheses for the two-sample location problem, Journal of Statistics Education 7 (2): www.amstat.org/publications/jse/secure/v7n2/callaert.cfm.

Mann HB and Whitney DR 1947 On a test of whether one of two random variables is stochastically larger than the other, The Annals of Mathematical Statistics 18 (1): 50-60.

The Wikipedia page for the Mann-Whitney U test can be accessed here, and the page for the Hodges-Lehman estimator is here.

For an online calculator of the Mann-Whitney U test you can visit Vassar’s page here.

For the critical values of the Mann-Whitney U test for samples sizes up to n1 = n2 = 20 and α = 0.05 or 0.01, see here.


Expanded sample for lognormal distribution

 Cinemetrics, Film Analysis, Film Studies, Film Style, Hollywood, Silent cinema, statistics  No Responses »
Apr 072011
 

I have looked at the assumption of lognormality for shot length distributions in the statistical analysis of film style in some earlier posts here, here, and here. Using the probability plot correlation coefficient, I concluded that as the assumption of lognormality could not be justified in up to half of the films studied that it was not appropriate to assume lognormality in general – if your experiment is based on assumption that is wrong 50% of the time, then your results will not be reliable. This post repeats that analysis presented earlier using a larger sample of Hollywood films. For a description of the method of using normal  probability plots and the probability plot correlation coefficient employed and the earlier results see the links above, or the links to the papers at the end of this post.

In Figure 1, we can see an example of a probability plot for a film (20,000 Years in Sing Sing) for which the data not only failed to reject the null hypothesis of lognormality (see below) but that on visual inspection is about as good a fit as you could expect. Although failure to reject a null hypothesis cannot be taken to imply that such a hypothesis is true, on the basis of this plot you would be more than happy to treat this film as being lognormally distributed.

Figure 1 Probability plot of shot length data (LN[X]) for 20,000 Years in Sing Sing (1932) (n = 692, PPCC = 0.9989)

Another useful feature of the normal probability plot is that the slope and the intercept of the regression line provide estimates of the shape factor and mean of the log-transformed data, respectively. In Figure 1, the slope is 0.8245, which is very close to the standard deviation of the log-transformed data (s = 0.8236) and the method proposed by Barry Salt for use in analysing film style based on the ratio of the mean to the median (s* = √(2×LN(mean/median)) = 0.8461). The intercept is 1.5695, giving a geometric mean of 4.7 seconds, which is very close to the median of 4.8s.

In Figures 2 to 4 we have three films for which the null hypothesis of lognormality (see below) was rejected. What is noticeable about these three films is that the deviation from the hypothesised distribution is different in each case. The plot for Rain (Figure 2) is all over the place; while the data for Steamboat Bill, Jr (Figure 3) deviates from the hypothesised distribution in both the lower and upper tails and the curvature of the plot indicates that the logarithmic transformation has not been successful in removing all of the skew from the data. In contrast, A Free Soul (Figure 4) shows such variation only in its lower tail.

Figure 2 Probability plot of shot length data (LN[X]) for Rain (1932) (n = 308, PPCC = 0.9768)

Figure 3 Probability plot of shot length data (LN[X]) for Steamboat Bill, Jr (1928) (n = 575, PPCC = 0.9839)

Figure 4 Probability plot of shot length data (LN[X]) for A Free Soul (1931) (n = 461, PPCC = 0.9873)

We can also see from the different estimates provide by the slope, s, and s*, and intercept and median that. For Rain, the slope gives a shape factor of 1.3036, s = 1.3287, and s* = 1.5865; while the intercept (1.8959) indicates a geometric mean of 6.7 compared to a median value of 5.1 seconds. For Steamboat Bill, Jr, the slope is 0.7135, compared to s = 0.7233 and s* = 0.8920; while the discrepancy between the geometric mean (5.2 [intercept = 1.6572]) and the median (4.8) is less than was observed for Rain. For A Free Soul the differences in the estimates are small, but nonetheless: for the shape factor, the slope is 1.0206, s = 1.0305, and s* = 1.0962; and that the geometric mean is 7.1 (intercept = 1.9596) and the median is 6.6 seconds.

Note that in all three cases, it is the method for estimating the shape factor based on the assumed relationship between the median and the mean (s*) that shows the greatest difference from the other methods. This is because the relationship between the median and the mean is only valid if the data is lognormally distributed. If this is not the case, then the claimed relationship between the median and the mean does not exist and produces inaccurate estimates of the parameters for the lognormal distribution. As this assumption is valid for 20,000 Years in Sing Sing we see that s* provides an estimate close to the other methods; but as the assumption of lognormality is not justified for the other three films, it does not. If we based any analysis of these films upon the assumption that their shot lengths were lognormally distributed, then our conclusions would be worthless because that assumption, and everything we derive from it (including the parameters μ and σ), is not true.

As noted above, it appears that the assumption of lognormailty may be justified in only half of the films we look at. Extending this research with a larger sample will allow us to make a better assessment of the applicability of this assumption to shot length distributions. In total, the probability correlation coeffcient test of normality was applied to a total of 168 Hollywood films (including some of the films I had previously looked at), divided into three groups: silent films of the 1920s (n = 52), sound films from 1929 to 1931 (n = 66), and sound films from 1932 to 1934 (n = 50). As these are statistical tests of a null hypothesis it is important to remember that failure to reject the null hypothesis does not mean that the data is lognormally distributed, and that some of these tests will conclude the data is not lognormally distributed when in fact it is. The test was applied using a Blom plotting position and α = 0.05. All the data used is from the Cinemetrics database (here).

Of the silent films produced in the 1920s (Table 1), the hypothesis of lognormality was rejected in 39 of the 52 cases, or 75% of the time. Of the sound films produced between 1929 and 1931 (Table 2), lognormality was rejected in 50 out of 66 cases (76%); and of the sound films from 1932 to 1934 (Table 3), it was rejected in 40 out of 50 cases (80%).

Table 1 Probability plot correlation coefficient test of the null hypothesis (H0) that the data is lognormally distributed for Hollywood films produced in the 1920s (n = 52)

Table 2 Probability plot correlation coefficient test of the null hypothesis (H0) that the data is lognormally distributed for Hollywood films produced from 1929 to 1931 (n = 66)

Table 3 Probability plot correlation coefficient test of the null hypothesis (H0) that the data is lognormally distributed for Hollywood films produced from 1932 to 1934 (n = 50)

What stands out from these results is that the proportion of films for which there is sufficient evidence against the assumption of lognormality is similar for each group of films. The earlier results that indicated that lognormality could not be assumed in half the films now look over-optimistic – the assumption of lognormality may only be justified in between a fifth to a quarter of cases for Hollywood films. Certainly this is a long way off the assumption that lognormality of shot length distributions is generally true. Whether this is true for cinemas in other countries or other eras will have to wait for a later post.

The other thing to stand out is that there is no pattern among the films: we cannot distinguish between short films or features, silent films or sound, films from different genres or different studios, or by decade as being lognormal or not lognormal. We can say that the assumption of lognormality will be justified in some cases, but that in the overwhelming majority of cases this is not true. Additionally, as noted above, they will be different from an assumed lognormal distribution in different ways. Statistical studies of film style should be developed with this in mind.

References

Filliben JJ 1975 The probability plot correlation coefficient test for normality, Technometrics 17 (1): 111-117.

Looney SW and Gulledge TR 1985 Use of the correlation coefficient with normal probability plots, The American Statistician 39 (1): 75-79.

Vogel RM 1986 The probability plot correlation coefficient test for the normal, lognormal, and Gumbel distribution hypotheses, Water Resources Research 22 (4): 587-590.


Estimating shot length distributions

 Cinemetrics, Film Analysis, Film Studies, Film Style, Hollywood, Silent cinema, statistics  No Responses »
Mar 172011
 

One of the problems we encounter when researching film style is that different versions of the same film exist. For example, the discovery in Argentina in 2008 of a version of Fritz Lang’s Metropolis (1927) that was approximately 25 minutes longer than previously known versions. The official site for the restored version is here. This makes the statistical analysis of film style difficult, because we have to face the fact that the version of a film we are analysing may not be the film as it was produced.

We may come across different versions of different films for a variety of reasons:

  • Different versions of the same film may be released in different countries.
  • ‘Director’s cut’ versions raise the question as to what we should call the definitive version: I have five different versions of Bladerunner (Ridley Scott, 1982) on DVD – the original 1982 US theatrical release, the 1982 international theatrical release, the work print, and the 1992 Director’s Cut version, and the 2007 Final Cut. We could simply note which version our data represents and leave it at that. However, we are faced with the problem that if we wish to look at the shot length distributions of Hollywood movies in the early-1980s or the films of Ridley Scott, which version should we pick? Should we pick the version with the voice-over that uses shots left over from The Shining, or the one without the voice-over and the unicorn dream sequence? Does the fact that the film was re-edited for the 2007 cut invalidate it when it comes to looking at 1980s Hollywood, even though all the material used in this cut was shot in the early-1980s? Is the Final Cut version an example of Hollywood cinema from the early-1980s, or of the mid-2000s? Did Scott’s editing style change between 1982 and 2007 so that these two versions cannot be simply compared?
  • The version of a film released for home viewing is often different to the theatrical release due to the requirements of classification boards or censors. Another factor here may be corporate taste: historically, some home cinema outlets have edited their tapes to maintain a family friendly corporate image by removing scenes of gore, violence, and/or sex. Rather than distinguish between different versions we tend to treat the domestic and theatrical releases as being one and the same, when in fact our data may show some discrepancies.
  • Although it is unlikely to be a significant factor in the 21st century, pan-and-scan may affect the number of shots in a film.
  • When working with silent films it is often difficult to find compete versions of the films, and some frames, shots, scene, or even reels may be missing. We will therefore be working with only a partial data set. This problem can be compounded by the release of restored versions that are built up from several prints. It is highly likely that we have not the seen (and probably never will) the original version of many of the silent films that we take for granted on DVD.

There are also other sources of measurement error that can affect our research:

  • When dealing with wipes, dissolves, fades, irises, should we record the end of one shot and the beginning of the next at the beginning of the transition, the end of the transition, or in the middle of the transition. I always prefer the last of these options, and try to identify the middle frame of the edit, but I cannot speak for other researchers.
  • Identifying the correct running speed for silent movies is problematic. Silent movies we often shot at 18 or 20 fps, while we view and analyse them at 25/30 fps.
  • Data from the Cinemetrics database will also contain errors due to the performance of the researcher, and so the figures quoted will only ever be estimates.

It is wrong to state, as Barry Salt did recently (here), that we do not need to employ the full range of statistical methods and that such methods are ‘misleading’ and ‘irrelevant.’ It is necessary to deal with these issues in order to present the best analysis we can, and that means we need to be able to deal with the error present in our estimates. Even though the data we collect may be accurate to the frame, we will still have to deal with the existence of multiple versions, missing shots, different methods of data collection, etc. If you are going to analyse film style statistically, then at some point you are going to have to do some statistics.

This post focusses on the particular problem of dealing with silent films that have been restored. In 2009 and 2010 I added two posts to this blog looking at the shot length distributions of the Keystone films starring Charlie Chaplin (here and here). Since then, the BFI has released its Chaplin at Keystone DVD (see here). How do the shot length distributions of these restored versions compare to the original data I used in 2009? (For ease of understanding, when I refer to ‘original’ I mean the 2009 data and when I refer to ‘restored’ I mean data derived from the BFI DVD).

To date I have only looked at data from four films, though I hope to get around to the rest some time later this year. The four films are The Masquerader, The Rounders, The New Janitor, and Getting Acquainted. In the original data I removed the credit titles and the expository and dialogue title, and for the sake of consistency I have done so here with the restored versions. However, in the Excel file at the end of this post that includes the shot length data from the restored versions of these four films I have excluded the credit titles but left in the other titles as indicated by ‘T.’

The descriptive statistics of the original and restored versions of The Masquerader are presented in Table 1 and the empirical cumulative distribution functions are presented in Figure 1.

Table 1 Descriptive statistics of the original and restored versions of The Masquerader (Charles Chaplin, 1914)

From Table 1 we can see that the original estimate of the median shot length (3.7s [95% CI: 2.8, 4.6]) is consistent with the revised estimate (4.5s [95% CI: 2.7, 6.3]). However, there is a large difference in the dispersion of shot lengths as indicated by the increase in the upper quartile and the interquartile range. This indicates that the version of The Masquerader from which the original data is less consistent in the upper part of the distribution, although a two-sample Kolmogorov-Smirnov test indicates there is no statistically significant difference (D = 0.1485, p = 0.265).

Figure 1 Empirical cumulative distribution functions of shot lengths in the original and restored versions of The Masquerader (Charles Chaplin, 1914)

Looking at the same information for The Rounders (Table 2 and Figure 2), we note that there is a much larger discrepancy between the two versions of this film. The original estimate of the median shot length was 3.6s (95% CI: 2.5, 4.7), and the revised estimate is 5.0s (95% CI: 3.5, 6.5). Again there is a larger increase in the dispersion of shot lengths, and this is also more marked in the upper part of the distribution. Again, we find that a two-sample Kolmogorov-Smirnov test indicates there is no statistically significant difference  between the two distribution functions (D = 0.1922, p = 0.087).

Table 2 Descriptive statistics of the original and restored versions of The Rounders (Charles Chaplin, 1914)

Figure 2 Empirical cumulative distribution functions of shot lengths in the original and restored versions of The Rounders (Charles Chaplin, 1914)

There are no such large differences between the versions of The New Janitor (Table 3 and Figure 3). The medians are consistent, with only a small change in the estimate from 3.5s (95% CI: 2.4, 4.5) to  4.2s (95% CI: 3.2, 5.1). There is also a small increase in the interqaurtile range, and this is accounted for by the small difference between  the upper quartiles. However, this difference is not comparable to those observed in the cases of The Masquerader and The Rounders, and the cumulative distribution functions are indicates that the two versions have the same distribution of  shot lengths (Kolmogorov-Smirnov: D = 0.1184, p = 0.515).

Table 3 Descriptive statistics of the original and restored versions of The New Janitor (Charles Chaplin, 1914)

Figure 3 Empirical cumulative distribution functions of shot lengths in the original and restored versions of The New Janitor (Charles Chaplin, 1914)

The two versions of Getting Acquainted (Table 4 and Figure 4) show only a small difference in the upper quartile and the interquartile range, but otherwise the two sets of shot length data are consistent (Kolmogorov-Smirnov: D = 0.0622, p = 0.978). The original estimate of the median is 3.9s (95% CI: 3.3, 4.5) and the revised estimate is 4.0s (95% CI: 3.3, 4.7), so these are nearly identical.

Table 4 Descriptive statistics of the original and restored versions of Getting Acquainted (Charles Chaplin, 1914)

Figure 4 Empirical cumulative distribution functions of shot lengths in the original and restored versions of Getting Acquainted (Charles Chaplin, 1914)

Although I have looked at just four films here we can see that generally the difference in the median shot lengths is small for three of the films and would not substantially change how we interpret this information – though the increase in the dispersion of the upper part of the distribution for the restored version of The Masqueraders is a good example of why it is not enough to refer only to measures of location in the analysis of film style. We must also look at dispersion. The difference between the two versions of The Rounders will obviously lead us to reconsider our conclusions based on this data. Hopefully when I have finally completed transcribing the data for the other Chaplin Keystones from the restored version a clearer understanding of how to deal with different estimates of the shot elngths in a motion picture will emerge.

The shot length data for the restored versions of The Masquerader, The Rounders, The New Janitor, and Getting Acquainted can be accessed as an Excel 2007 (.xlsx) here: Nick Redfern – BFI Restored Chaplin 1. This data was collected by loading the films into Magix Movie Edit Pro 14 at 25 fps, and has been corrected by multiplying each shot length by 25/24.


Claude Hamilton Verity III

 British Cinema, Film History, Film Industry, Film Studies, Film Technology, Silent cinema  No Responses »
Mar 102011
 

I have drawn attention to the Leeds inventor Claude Hamilton Verity and his efforts to develop a sound-on-disc system for the synchronization of image and sound in two earlier posts that can be accessed here and here. This week I bring to your attention some other references to Verity I have come across recently.

First, an article by Frank H. Lovette and Stanley Watkins, titled ‘Twenty Years of “Talking Movies:” an Anniversary’ and published in the 1946 volume of Bell Telephone Magazine, refers to Verity as someone who made a notable contribution to the development of talking pictures alongside such illustrious names as Thomas A. Edison, Pathé Freres, Leon Gaumont, and Orlando E. Kellum. The article can be accessed here.

The authors clearly do not take The Jazz Singer in 1927 to be point at which pictures began to talk, and instead choose as their starting point the demonstration of the Vitaphone system on 6 August, 1926, for the screening of Don Juan, starring John Barrymore. This is unsurprising given that Bell was itself involved in the development of this system, but they do describe this screening somewhat poetically:

Before the applause could die away, the dramatic sequences of Don Juan unfolded against their synchronized musical background. Scientists, public officials, prominent figures from many walks of life sat in amazement until the last crescendo and finale of this scientific marvel. The men who brought it into being by their refinement of existing arts were hailed as having made possible “the greatest invention of the twentieth century.” And Dr. Michael I. Pupin was led to exclaim that “no closer approach to resurrection has ever been made by science.” The pioneers of Western Electric and Bell Telephone Laboratories and their collaborators of Warner Brothers and Vitaphone experienced that night a measure of accomplishment which few men of science ever live to taste or see.

We can forgive the authors a touch of hyperbole when writing about Bell-developed technology in a Bell-funded journal, but this raises an interesting question about when we should date the earliest successful demonstration of synchronized sound in cinema. There were other inventors to successfully demonstrate the synchronization of sound and image prior to 1926, including Kellum’s Photo-kinema system and Verity’s system both of which were demonstrated in 1921. D.W. Griffith used the Photo-kinema system for Dream Street, which premiered on 2 May, 1921, with two sound segments; and we have reports of the demonstration of two original shorts produced by Verity in Harrogate on 30 April, 1921 (see the first link above). We also know that in November 1923, Verity sailed to New York to meet with J. Stuart Blackton of the Vitagraph Film Company and gave an interview to The New York Times regarding the synchronization of sound and image in January 1924 (see the second link above). We do not know what impact Verity’s work in England had – if any – on the development of ‘the greatest invention of the twentieth century.’

The article refers to Verity’s system as Veritiphone, but this term appears only infrequently in other articles.

Second, two articles in the Wellington Evening Post from 1921 and 1923 refer to Verity’s efforts. These articles are available from Papers Past at the National Library of New Zealand, and rather wonderfully, they can be reproduced under a creative commons licence.

The first article was published on 3 June, 1921, and is largely a direct quote from an earlier article published in The Daily Mail. I have not found this earlier article, but given the timing I assume that the demonstration referred to was the one that took place in Harrogate in April 1921.

TALKING FILMS

PERFECT VOICE MOVEMENT CLAIM

Talking kinema films, it is claimed, have definitely advanced a stage as ‘the result of the invention of a synchroniser by Mr. Claude H. Verity, a Harrogate engineer. With this instrument in the projector-box, it is stated, an operator, by simply sliding a knob quite independently of watching the screen, can work synchronisation to 1-24th of a second.

In a Harrogate building where secrecy has been maintained for nearly five years of experimenting, writes a correspondent to the London Daily Mail, who has witnessed a straight drama and cross-talk comedy exhibited in conjunction with a gramophone. “There was no mistaking the accuracy of voice and lip movement. If it should vary a tenth of a second it would be due to the fact that the actors were so much out in repeating for the gramophone recorder what they had done for the screen. These processes are separate and are linked up by an expert stenographer.

“The synchroniser does away with the necessity for stopping the action of a picture to introduce worded explanations; indeed, dialogue becomes a distinct part of the picture.

“For operas with singing and music a child could work it because there is a fixed tempo. Should the film break the speaking can be stopped and taken up again.”

A great advantage of the invention, it is urged, is that with the apparatus in projecting-boxes the synchronised film could be circulated in the ordinary way.

The two films referred to above would be The Playthings of Fate (the drama) and A Cup of Beef Tea (the comedy). I would assume that this is the first time the term ‘cross-talk comedy’ is used in reference to the cinema.

The second article was published on 1 September 1923, and is only a passing reference to Verity as part of a much larger piece.

Synchronization of the film and its musical counterpart seems to be solved by the “Veritphone,” an invention of Claude H. Verity, of Leeds, England. It aims at the alliance of sound and movement by the combination of a double set of “super-gramophones,” and an ingenious indicator, which shows when the film and the sound record are together.

Details of Verity’s patents that give a more detailed explanation of how the system worked can be accessed in my earlier posts.

Third, and slightly confusingly, there is another reference to Verity derived from an article published in The Daily Mail in De Sumatra Post published on 11 November 1922. I have no idea what this says because it is in Dutch. The complete issue of De Sumatra Post can be downloaded as a pdf file here (it’s about 7.1 MB and I think it is from the Dutch equivalent of Papers Past), and the short piece referring to Verity is at the bottom of page 14.

Fourth, a notice in The Electrical Review 90 1922: 416 announces the successful demonstration of Verity system at the Albert Hall in Leeds in 1922 (the date is given as 3 March whereas other articles give the date as 3 April), noting that

By experiment over a considerable time past, Mr. Verity has provided an apparatus which certainly yields co-timing of the lip movements of the persons on the screen with the sounds emitted from the electrically-controlled gramophone, …

By the time of his 1922 demonstrations, Verity had spent at least 5 years and (by his own estimation) some £7000 of his own money developing his synchronisation system.

Finally, and a good deal less wonderful than anything from New Zealand, is a reference to Verity in an article published in Political Science Quarterly in 1948. The full reference is Swensen J 1948 The entrepreneur’s role in introducing the sound motion picture, Political Science Quarterly 63 (3): 404-423. I do not know how this piece refers to Verity – it may be only as a name in a footnote, possibly derived from the Bell Telephone Magazine article referred to above – because the article lies behind a paywall at JSTOR. There is no good reason why an article from 1948 should be behind a paywall in 2011.

 

 


Some notes on cinemetrics IV

 Cinemetrics, Film Analysis, Film Studies, Film Style, Laurel and Hardy, Silent cinema, statistics  No Responses »
Nov 252010
 

In the 1970s, Barry Salt proposed that the mean shot length could be used to describe and compare the style of motion pictures. Many other scholars have followed him, and we find now that average shot lengths are now commonly cited in film studies texts. Unfortunately, a worse choice of a statistic of film style could not have been made – the distribution of shot lengths is not normally distributed and the mean does not accurately locate the middle of the data. This means that a large part of film studies research is utterly useless because it is based on an elementary mistake in the methodology that could have been avoided with only a middle school maths education. Quite simply, the mean is not an appropriate measure of location for a skewed dataset with a number of outliers. It never has been; it never will be; and quoting this as a statistic of film style leads to fundamentally flawed inferences about film style, as can be seen here.

This does not mean tha Salt has decided to give up on the mean shot length. He has subsequently asserted – but not proven – that shot length distributions are lognormally distributed, and that the mean shot length should be retained because the ratio of the mean shot length to the median shot length can be used to derive the shape factor of a lognormal distribution that adequately describes the distribution of shot lengths in a motion picture. (Actually Salt refers to the median-to-mean ratio, but this is just a different way of writing the same information – each ratio is reciprocal of the other. For convenience in later calculations I refer only to the mean-to-median ratio). The ratio of the mean to the median is a measure of the skew of a dataset – symmetrical distributions have a ratio of approximately 1 – and is used widely in economics to represent imbalances in income. If a distribution is lognormal, there is a relation between the mean-to-median ratio and the shape factor of a lognormal distribution. As I have shown elsewhere on this blog, the assumption of lognormality is not justified – applying a normality test to the log-transformed data I have found that the null hypothesis of lognormality is rejected in between 50% and 80% of cases. The proportion of silent films for which this null hypothesis is rejected appears to be greater than the proportion of sound films.

Undeterred, Salt persists with the assertion that shot lengths are lognormally distributed and has cooked up a new scheme to justify this assertion by arguing that titles should be removed from the shot length data of silent films and then analysed as being lognormal. No suggestion is made regarding the seemingly large proportion of sound films that also do not appear to be lognormally distributed. As is typical in Salt’s work, this argument is simply asserted as being true without any methodological justification and – as we shall see – some dubious evidence.

What is the methodological justification for removing the titles from the shot length data? Possible reasons for removing this data are that the titles are not original and have been updated so that they no longer accurately reflect the original structure of the film. However, the fact that the titles may not be original does not automatically mean that the titles are inaccurate or that their time on screen is not an accurate reflection of the original tempo of the film. It may be that a conservator has meticulously restored the film and respected the way the film was originally put together. We should certainly feel free to include the titles in the data if they are or are known to be properly restored, are based directly on the original film, or are reasonable approximations based on documentary evidence for the film’s production, historical context, etc. Salt’s suggestion appears to be a blanket ban on all titles in shot length data for silent films, but this would rule out much otherwise useful data. A further appears in the memoir of the projectionist  Louis J Mannix (whom I discussed in an earlier post), who noted that it was a practice of projectionists to slow the film when a title came onto the screen for the ease of reading by the audience – there is nothing we can do as statisticians to control for this type of situation specific variability but it is very interesting as film history. The use of titles is certainly a methodological concern for analysts of film style, and it does need to be discussed as part of the methodology of the statistical analysis of film style. This would, however, mean going beyond mere assertion.

Salt’s method involves linking two shots that were previously separated by a title into a single shot, but again there is no methodological justification for this. The decision to put a title in the middle of a shot is itself an aesthetic decision by the filmmakers for the purposes of narrative communication, and should be respected as such. If we combine the shots in the manner Salt suggests can the data be said to reflect the film as it was made? The tempo of the film is changed, and we can no longer make any direct comparison between silent films, and between silent films and sound films. Salt also states the resulting analysis will provide very different results if the shots are not combined in this way, but he does not say why we should prefer his method over the alternative of not combining the shots.

Separating titles from the rest of the shot length data for a film is not in itself a bad idea – it would allow us to look more closely at how a film was put together, and to make inferences about how audiences understand silent films or text on screen in general. However, Salt appears to want to remove this data to make it fit a lognormal distribution, and that is a bad idea. It is back to front: the transformation of the data is suggested to make it fit a preconceived theoretical distribution, even though there is no evidence that this assumption is justified in general. If the method of combining shots is to be preferred to not combining them for the purpose of generating a better lognormal fit, then this is clearly problematic. In the absence of a proper methodological basis, this smacks of both desperation and data manipulation. Nonetheless, Salt has stated that this approach can be termed ‘experimental film analysis’ similar to experimental archaeology. The whole thing can be read here.

Little Annie Rooney has been held up of an example of how the fit to a lognormal distribution is improved after removing the titles. The data for this film (without titles) is here. However, closer examination of the data reveals that the mean-to-median ratio leads to a poor estimate of the shape factor and provides a substantially poorer fit than the maximum likelihood estimates (MLE). Recalling that a random variable X (such as the length of a shot) is lognormally distributed if its logarithm is normally distributed, Figure 1 presents the histogram of the shot length data transformed using the natural logarithm and three density estimates.

Figure 1 Density estimation of shot lengths for Little Annie Rooney (minus titles)

The red curve is the kernel density estimate, using an Epanechnikov kernel and a bandwidth of 0.5, and is a nonparametric density estimate that makes no assumption about the shape of the distribution and depends on the data alone. This is the empirical distribution of the log-transformed data, and is used as a part of exploratory data analysis. From the histogram and the kernel density estimate we can see that even after the data has been log-transformed there is still some skewness and a heavy upper tail. We should therefore be sceptical about the assertion that this data is lognormally distributed. (For a kernel density calculator see here).

The black curve is the normal distribution specified by the maximum likelihood estimators of the log-transformed shot lengths – i.e. the mean (μ) and standard deviation (σ) of the logarithms of the shot lengths. (Note that μ is the arithmetic mean of the log-transformed data and the geometric mean of the data in its original scale). For this data, μ = 1.2078 and σ = 0.7304. The probability plot correlation coefficient (PPCC) using a Blom plotting position is 0.9776 and the null hypothesis that the data (n = 1066) is lognormally distributed is rejected for α = 0.05. Figure 2 is the normal probability plot for this data with the parameters of the black curve. (Recall that if the lognormal distribution is a good fit, the data will lie along the red line).

Figure 2 Normal probability plot for Little Annie Rooney (minus titles): LN[X]~N(1.2078, 0.7304)

The green curve in Figure 1 is the normal distribution defined if we take the median shot length and the estimate of σ derived from the mean-to-median ratio, as Salt recommends. According to Salt, the mean-to-median ratio for shot length data is equal to the exponentiate of half the variance (μ/med = exp (σ2/2)) and that from this we can estimate σ. As we know the value of σ is 0.7304, this can be tested for Little Annie Rooney. The ratio of the mean-to-median ratio for this film is 4.6/2.9 = 1.5862 and exp (0.73042/2) = 1.3057. The mean-to-median ratio overestimates the true value by 21.5%. Inevitably, this leads to a poor estimate of σ: if μ/med = exp (σ*2/2) then σ* = √ (2 × LN (μ/med)), and for Little Annie Rooney (minus titles) this produces an estimate of σ* = 0.9606. (It is perhaps not clear from the font used here, but √ is ‘square root’). The estimated value of the shape factor is greater than its MLE value by 31.5%. Looking at the function of LN[X]~N(1.0647, 0.9606) in Figure 1, we can see that it provides a better fit to upper tail of the data and is very close to the kernel density estimate. At the same time, it provides a very poor fit below the median, and is actually worse than the MLE parameters. This can be seen more clearly by looking at Figure 3, which is the normal probability plot assuming LN[X]~N(1.0647, 0.9606). (This already poor fit can be made worse by substituting μ for the median).

Figure 3 Normal probability plot for Little Annie Rooney (minus titles) LN[X]~N(1.0647, 0.9606)

From this we can conclude that (1) the shot length data for Little Annie Rooney (minus titles) is not lognormally distributed; (2) that the mean-to-median ratio does not equal exp (σ2/2); and (3) that using the mean-to-median ratio to derive σ* provides a very poor estimate of the shape factor. (Conclusion 1 should also lead us to question the method by which Salt claims to measure goodness of fit).

This same process cannot be applied to shot length data available of the Cinemetrics website for Little Annie Rooney with titles, as this data includes a shot length (presumably rounded down) of 0.0 seconds. (The logarithm of X ≤ 0 does not exist). This shot length does not appear in the data after the titles have been removed, and I find it hard to believe that this film had a title card that was present on screen for less than 0.05 of a second. The accuracy of this data with or without titles is questionable.

If we examine the shot length distributions of the silent short films of Laurel and Hardy (both with and without titles) we again find that (1) the assumption of lognormality is not justified, (2) the mean-to-median ratio does not provide reasonable estimates of exp (σ2/2), and (3) σ* does not provide reasonable estimates of σ.

Calculating the probability plot correlation coefficient for these films with titles using a Blom plotting position and α = 0.05, the null hypothesis that the data is lognormally distributed is rejected for 10 of the 12 films. Repeating this process with the titles removed, the null hypothesis is rejected for 11 films. (Recall that a statistical hypothesis test is a test of plausibility of the null hypothesis for a given set of data – failure to reject the null hypothesis indicates only that there is insufficient evidence to reject [and does not prove] H0). These results are presented in Table 1. The assumption of lognormality is not justified and removing the titles from the data does not affect this conclusion.

Table 1 Probability plot correlation coefficient for the silent films of Laurel and Hardy with and without titles

Table 2 includes the mean, median, and the standard deviation of the log-transformed data (σ). Using this information, we can test Salt’s other claims regarding the mean-to-median ratio. (Actually this is all rather redundant as we already know that lognormality is not a plausible model for this data). Early to Bed was excluded from this part of the study as the log-transformed data exhibits bimodality.

Table 2 Mean, median, and σ for the silent films of Laurel and Hardy with and without titles

First, let us ask if the mean-to-median ratio is equal to exp (σ2/2) for these films. The results are presented in Table 3, and it is immediately clear that for only two films – The Second Hundred Years and Angora Love – does μ/med provide a reasonable estimate of exp (σ2/2) when we include the titles in the data, and the PPCC test failed to reject the null hypothesis of lognormality for both these films. For every other film, μ/med overestimates the true value by ~10% or more. Once the titles are removed, we do not get the improvement Salt claims will be evident by censoring the data in this way. Generally, the change in the estimate once the titles are removed is small, although both The Second Hundred Years and Angora Love show much larger errors after the data has been censored due to an increase in the skew of the data.

Table 3 Mean-to-median ratio and exp (σ2/2) for the silent films of Laurel and Hardy with and without titles

Table 4 presents the maximum likelihood estimate of σ and the estimate derived by using σ* = √ (2 × LN (μ/med)) for the Laurel and Hardy films, both with and without titles. For the shot length data including titles σ* provides a poor estimate for those films that rejected the null hypothesis of lognormality in the PPCC test, and consistently overestimates σ by at least 12%. Again this is not surprising, as μ/med = exp (σ2/2) is only valid if the data are lognormal, which is not the case here. Turning to the shot length data after the titles have been excluded, we see that σ* is a poor estimate of σ for all the films in the sample.

Table 4 σ and σ* for the silent films of Laurel and Hardy with and without titles

From these results we can conclude that:

  • The methodological justification for removing titles from the shot length data of silent films is obscure, and lacks a theoretical basis.
  • There is no evidence to justify the assumption of that shot length data is lognormally distributed.
  • There is no evidence that removing the titles from silent films will improve the fit to a lognormal distribution, and may in fact produce a poorer fit.
  • The mean-to-median ratio does not provide a good estimate of exp (σ2/2).
  • Using the mean-to-median ratio to estimate the shape factor does not provide relaible results.

In other words, the approach suggested by Salt is wrong in every possible way.

Do not take my word for it. Do not blindly accept what someone tells you with scientific sounding words no matter how confident they sound. Learn to do it for yourself – it really is not that difficult to pick up enough statistics to be able to properly evaluate a research paper. Get some data and do your own testing. If you still get stuck then ask a statistician.

If you want to repeat the Laurel and Hardy tests performed above, I have added a spreadsheet to the Laurel and Hardy post (here) that includes the data with titles indicated.


Shot length distributions in the films of Laurel and Hardy

 Cinemetrics, Film Analysis, Film History, Film Studies, Film Style, Film Technology, Hal Roach, Hollywood, Laurel and Hardy, Oliver Hardy, Silent cinema, Stan Laurel  No Responses »
Oct 212010
 

This week I put up the first draft of my analysis of the impact of sound technology on the distribution of shot lengths in the short films of Laurel and Hardy from 1927 to 1933. The pdf file can be accessed here: Nick Redfern – Shot length distributions in the short films of Laurel and Hardy.

Abstract

Stan Laurel and Oliver Hardy were one of the few comedy acts to successfully make the transition from the silent era to sound cinema in the late-1920s. The impact of sound technology on Laurel and Hardy films is analysed by comparing the median shot lengths and the dispersion of shot lengths of silent shorts (n = 12) produced from 1927 to 1929 inclusive, and sound shorts (n = 20) produced from 1929 to 1933, inclusive. The results show that there is a significant difference (U = 56.0, p = 0.0128, PS = 0.2333) between the median shot lengths of the silent films (median = 3.5s [95% CI: 3.2, 3.7]) and those of the sound films (median = 3.9s [95% CI: 3.5, 4.3]); and this represents an increase in shot lengths in the sound films by HLΔ = 0.5s (95% CI: 0.1, 1.1). The comparison of Qn for the silent films (median = 2.4s [95% CI: 2.1, 2.7]) with the sound films (median = 3.0s [95% CI: 2.6, 3.4]) reveals a statistically significant increase is the dispersion of shot lengths (U = 54.5, p = 0.0109, PS = 0.2271) estimated to be HLΔ = 0.6s (95% CI: 0.1, 1.1). Although statistically significant, these differences are smaller than those reported in other quantitative analyses of film style and sound technology, and this may be attributed to Hal Roach’s commitment to pantomime, the working methods of Laurel, Hardy, and their writing/producing team, and the continuity of personnel in Roach’s unit mode of production which did not change substantially with the introduction of sound.

UPDATE: 25 November 2010. WordPress have now very helpfully made it possible to upload Excel spreadsheets to blogs, and so I have replaced the Word file with an Excel file that is much easier to use. This data also now includes information of which shots are titles (as idicated by a T in an adjacent column). I accept no libaility for any problems you may have when downloading and using Excel spreadsheets on you computer. The data used in this study can be accessed in the form of an Excel .xls file here: Nick Redfern Laurel and Hardy shot lengh data. The methodology behind the sources and collection of this data is described in the above paper.