| Basic Commands | Probability Puzzles | Hypothesis Test, Count Data | Hypothesis Test, Measured Data | Confidence Interval, Count Data | Confidence Interval, Measured Data | Association / Correlation | Regression | Other Examples |

Reinforce-1

[Friedman two-way analysis of variance; see REINFORCE-2 for a variance of column ranks version]

Problem

Grosslight and Radlow studied the effect of different patterns of reinforcement on discrimination learning in rats:

[T]hree matched samples (k=3) of 18 rats (N=18) were trained under three patterns of reinforcement. Matching was achieved by the use of 18 sets of litter mates, 3 in each set. Although all of the 54 rats received the same quantity of reinforcement (reward), the patterning of the administration of reinforcement was different for each of the groups. One group was trained with 100 percent reinforcement (RR), a second matched group was trained with partial reinforcement in which each sequence of trials ended with an unrewarded trial (RU), and the third matched group was trained with partial reinforcement in which each sequence of trials ended with a rewarded trial (UR).

After this training, the extent of learning in rats was measured by the speed at which . . . rats learned an "opposing" behavior -- whereas they had been trained to run to white, [they] were now rewarded for running to black. The better the initial learning, the slower should be this transfer of learning. The prediction was that different patterns of reinforcement would result in differential learning as exhibited by ability to transfer.

Reinforce -1 Table. Rankings of Learning Rates Of 18 Matched Groups Of Rats Under Three Conditions of Reinforcement

Type of reinforcement
Group	rewarded/rewarded (RR)	rewarded/unrewarded (RU)	unrewarded/reawrded (UR)
1	1	3	2
2	2	3	1
3	1	3	2
4	1	2	2
5	3	1	3
6	2	3	2
7	3	2	1
8	1	3	2
9	3	1	2
10	3	1	2
11	2	3	1
12	2	3	1
13	3	2	1
14	2	3	1
15	2.5	2.5	1
16	3	2	1
17	3	2	1
18	2	3	1
rank sum	39.5	42.5	26.0

Note. Data are adapted from Siegel & Castellan, 1988, Table 7.4, p. 179.

...Null hypothesis (H₀): the different patterns of reinforcement have no differential effect on the observed behavior. Alternative hypothesis (H₁): the different patterns of reinforcement have a differential effect. (Grosslight & Radlow, 1956, cited in Siegel & Castellan, 1988, pp. 177-178)

If the null hypothesis is correct, then the rankings should be more or less equal for all three treatments, so the sum of ranks for each treatment (the column sums) should be approximately equal. How divergent could these column sums be purely by chance? Siegel and Castellan chose the Friedman two-way analysis of variance by ranks. This test involves calculation of the sum of squares of the three sums of ranks. Then the sum is multiplied by a coefficient depending on k and N, and another coefficient (depending on k and N) is subtracted. The "observed" value of the test statistic is to be compared to a table derived by analytical method to determine the probability of getting, by chance, a Friedman test statistic value that is equal to or greater than the observed value.

We will use resampling to determine this probability. Because only the rank sum depends on the data, and resampling is used, we may use this sum as the test statistic. We can omit calculation of the coefficients depending only on k and N - their purpose is to scale the sum of squares to a value that can be compared to a standardized table. Here is the sum of squares of the three sums of ranks:

R1^2 + R2^2 + R3^2 = 39.5^2 + 42.5^2 + 26.0^2

= 1560.25 + 1806.25 + 676.0

= 4042.

Resampling Procedure

1. Prepare 17 groups of three cards; number the cards in each group 1, 2, and 3. Prepare another group of three cards labeled 1, 2.5, and 2.5 (for the one case that had a tie).
2. Shuffle the cards in each of the 18 groups and arrange in three columns at random: Each row contains cards from one group. Compute the sums of the three columns.
3. Compute the sum of squares of the three column sums. If the sum is equal to the "observed" value (4042.5) or greater, record "yes"; otherwise record "no."
4. Repeat (2-3) 1,000 times and determine the proportion of "yes."

Computer Implementation in Resampling Stats

COPY (1 2 3) ranks

set up vector "ranks" in preparation for the simulation

COPY (1 2.5 2.5) ranks2

for one set of rats, there was a tie, hence a need for explicit handling of that row of data

COPY (0) null

We will use "null" later for the SUMSQRDEV command. In effect, the null vector will become 0 0 0

REPEAT 1000
  SET 3 0 sums$

Initialized a vector of cumulative sums. The <$> signifies a simulation vector.

  REPEAT 17

there were 18 groups of rats, but 1 group had a tie so is handled separately

    SHUFFLE ranks ranks$

shuffle the numbers 1, 2, and 3

    ADD sums$ ranks$ sums$

add the numbers to the vector of sums$

  END
  SHUFFLE ranks2 ranks2$

Special handling for rat group #15, which had a tie. Shuffle the numbers 1, 2.5, and 2.5.

  ADD sums$ ranks2$ sums$

and add these numbers to the vector of sums

  SUMSQRDEV sums$ null sqsums$

This command takes the differences between the three values in "sums$," subtracts (0 0 0) of vector "null," and sums the results. The effect is to square each value in "sums$" and add them together.

  SCORE sqsums$ scrboard

record the value

END
COUNT scrboard >=4042.5 yes

determine how often the simulation produced a Friedman statistic as high or higher than the experimental result

DIVIDE yes 1000 prob

convert to a proportion of the 1,000 repeats

convert to a proportion of the 1,000 repeats 

Results

Results of 3 runs:

prob = 0.016 

prob = 0.014 

prob = 0.011

Conclusion

We conclude that the different patterns of reinforcement have a differential effect. In statistical terms, we reject the null hypothesis -- that the patterns have no effect -- at significance level p of about 0.015. Siegel and Castellan arrived at the same conclusion and reported a significance level at between 0.02 and 0.01.

References

Grosslight, J.H., & Radlow, R. (1956). Patterning effect of the nonreinforcement-reinforcement sequence in a discrimination situation. Journal of Comparative and Physiological Psychology, 49, 542-546.

Siegel, S., & Castellan, N. J., Jr.. (1988).

Nonparametric statistics for the behavioral sciences (2nd ed.). New York: McGraw-Hill.