A Constant Entropy Increase Model for the Selection of Parallel Tempering Ensembles

The present paper explores a simple approach to the question of parallel tempering temperature selection. We argue that to optimize the performance of parallel tempering it is reasonable to require that the increase in entropy between successive temperatures be uniform over the entire ensemble. An estimate of the system’s heat capacity, obtained either from experiment, a preliminary simulation, or a suitable physical model, thus provides a means for generating the desired tempering ensemble. Applications to the two-dimensional Ising problem indicate that the resulting method is effective, simple to implement, and robust with respect to its sensitivity to the quality of the underlying heat capacity model. © 2008 American Institute of Physics . (cid:3) DOI: 10.1063/1.2907846 (cid:4)


I. INTRODUCTION
Stochastic quadrature methods are valuable tools in the study of many-body systems.In particular, they offer a general approach to broad classes of both classical 1 and quantum-mechanical problems. 2A common practical issue associated with the application of these methods is coping with the "sparse" probability distributions that accompany activated or "rare-event" processes.Unless special care is exercised, the random walk procedures typically used to implement such stochastic approaches can become trapped in the myriad of the system's local potential minima or "inherent structures," thereby producing an inefficient, or even improper sampling of the underlying probability distribution.
][5][6] The essence of these procedures is to create an ensemble of replicas of the system of interest corresponding to a range of one or more control parameters and to utilize the resulting ensemble to improve the sampling.For example, if the tempering control parameter involved is the system's temperature, the strategy is to use information from the ensemble's high-temperature members, where activation barriers are more easily surmounted, to improve the sampling at lower temperatures, where barrier crossings are otherwise exponentially suppressed.In practice, this is accomplished by introducing attempted exchanges of configurations between ensemble members.For these attempted moves to be useful, the temperature range of the ensemble must be sufficiently large that its higher energy members can surmount relevant activation barriers and the spacing must be sufficiently small that exchange attempts between the various temperatures are statistically significant.
The efficiency of the parallel tempering method depends on the details of the control parameter selection.A number of techniques for guiding this selection are available and have been reviewed elsewhere. 6,7One fruitful line of the development is represented by the works of Kofke, 8 Kone and Kofke, 9 and Predescu et al., 10 who have explored connections between the success of tempering exchanges and the heat capacity of the system being studied.More recently, Predescu et al. 11 have explored ensemble selection schemes based on alternative performance metrics such as the "effective fraction."][19] The present paper examines a simple, intuitive approach to the question of parallel tempering temperature selection.As in the works of Kofke, 8 Kone and Kofke, 9 and Predescu et al., 10 the system's heat capacity plays a central role.After introducing the basic idea in Sec.II, we examine its relationship to existing methods.In Sec.III, we compare the performance of various temperature selection techniques when applied to a common problem.

II. FORMAL DEVELOPMENTS
In its most basic form, conventional parallel tempering involves the creation of an ensemble of equilibrium simulations of the physical system in question that covers a range of temperatures, denoted here by ͕T m ͖, m =1, M + 1. Sampling moves involve a mixture of conventional Metropolis displacements at the individual temperatures as well as attempted exchanges of information between different data streams.One has great freedom in selecting both the range of temperatures as well as the details of the exchange attempts.Indeed, it is guidance concerning the selection of such procedures that we seek.a͒ Electronic mail: jimmieគdoll@brown.edu.Kofke, 8 Kone and Kofke, 9 emphasize the notion that the probability of accepting tempering exchange attempts between different ensemble members is related to the entropy differences involved.To optimize the performance of parallel tempering, we suggest that it is reasonable to supplement this idea and to require that the increase in entropy ͑or "information"͒ between successive temperatures be uniform over the entire ensemble.That is, if ⌬S is the overall entropy difference between the minimum and maximum temperatures of the ensemble, T 1 and T M+1 ͑presumed known͒, the suggestion is to require the entropy increase between each successive pair of temperatures, T m and T m+1 to be a constant, ⌬S / M, for all m.Explicitly, assuming a constant volume, This choice of temperatures has the desirable feature that each successive temperature stream in the ensemble bears the task of producing a fixed amount of "new" information, a seemingly equitable way of "sharing" the computational workload between the ensemble members.It is clear that this constant entropy increase ͑CEI͒ approach can be implemented for general, temperature-dependent heat capacities.
Given the overall temperature range, the number of ensemble replicas to be used, and an estimate of the system's heat capacity ͑obtained either from experiment, a preliminary simulation, or a suitable physical model͒, Eq. ͑1͒ provides a means for generating the desired tempering ensemble.Specifically, one first integrates the model heat capacity from the minimum to maximum temperatures, T 1 and T M+1 , to obtain the overall entropy change ⌬S, and then solves for the successive temperatures T 2 , T 3 , ... ,T M that satisfy Eq. ͑1͒.Depending on one's wishes, this approach can be implemented either as a single-step or as an iterative procedure in which successive estimates of the system's heat capacity are utilized to refine the quality of the ensemble selection.
Beyond the question of "how" to select the ensemble temperatures, one must also consider the question of "how many" temperatures to include.A reasonable estimate can be obtained by requiring that entropy difference between successive ensemble members be less than or equal to the natural entropy fluctuations induced by energy fluctuations at those individual temperatures.Such a requirement assures at least a minimal similarity between the configurations sampled at the two different temperatures, and thus, a reasonable chance that tempering exchange attempts between those temperatures will be successful.Recalling that the increase in entropy with respect to energy at fixed volume is given by and denoting the relevant energy fluctuations at a given temperature as E , a reasonable choice of M is thus to require that the inequality be satisfied at each of the temperatures in the ensemble.
Expressing the energy fluctuations in terms of the system's ͑generally temperature dependent͒ heat capacity, Eq. ͑3͒ becomes For a given value of ⌬S, we thus want to select a value of M that is large enough to assure that Eq. ͑4͒ holds at each of the temperatures in the ensemble.
The CEI approach provides a general means by which we can select the parallel tempering ensemble.Under particular circumstances, these general results simplify significantly.For example, if the heat capacity is relatively constant over a particular temperature interval, Eq. ͑1͒ suggests that the differences between successive temperatures be taken as as suggested by Nymeyer et al. 20 on the basis of somewhat different arguments.If, in addition, the heat capacity is constant over the entire interval, Eq. ͑5͒ produces a geometric distribution for the temperatures in the parallel tempering ensemble.

III. NUMERICAL EXAMPLE
In this section, we consider the application of the CEI approach to a specific example, the zero-field, twodimensional ͑2D͒ Ising model. 21We choose this model both to illustrate the use of the present technique on a wellcharacterized problem and to facilitate the comparison of the CEI method's performance to that of other, recently published approaches. 14,16With the latter goal in mind, we utilize a tempering ensemble that consists of 21 temperatures covering the reduced temperature range of T = ͑0.1,10.0͒ throughout.
The exact, analytical heat capacity of the 2D Ising system 21 is plotted in Fig. 1 as a function of T. When used in the CEI model, this heat capacity produces the 21 member parallel tempering ensemble depicted in Fig. 2. In this figure, the temperature index N͑T͒, which is an integer index ranging from 1 ͑lowest temperature in the ensemble͒ to 21 ͑highest temperature in the ensemble͒, is plotted as a function of T. As can be seen by comparing Figs. 1 and 2, within the CEI model, a maximum in the heat capacity leads naturally to a "clustering" of the parallel tempering members in the corresponding temperature range.We note also that the strong temperature dependence of the heat capacity for the 2D Ising problem produces an ensemble that is far from geometric in nature.
The system's exact, analytical heat capacity will typically not be available for general applications.Consequently, it is important to assess the CEI method's sensitivity to the quality of the heat capacity model used to drive the approach.Does the approach retain its utility when a heat ca-pacity obtained from approximate and/or numerical means is used as the basis for ensemble selection?In Fig. 3, we show the exact 2D Ising model heat capacity of Fig. 1 ͑dashed line͒ along with the approximations numerically obtained from Monte Carlo simulations that utilize 20ϫ 20 ͑solid circles͒ and 4 ϫ 4 ͑open circles͒ periodic lattice grids, respectively.The CEI tempering ensembles generated by using the various estimates of the system's heat capacity are presented in detail in Table I.The various CEI ensembles are labeled according to the heat capacity model used to produce them.For example, the CEI-S͑4͒ ensemble is obtained by using the numerical 4 ϫ 4 grid estimate of the system's heat capacity to drive the CEI approach, the CEI-S͑20͒ using the numerical 20ϫ 20 grid estimate, and so on.The figure shows the analytical results for the infinite lattice ͑dashed line͒, for a 20ϫ 20 grid ͑solid circles͒, and for a 4 ϫ 4 grid ͑open circles͒.These heat capacity results in conjunction with the CEI method generate the parallel temperature ensembles listed in Table I. 21-temperature ensemble for the current problem.The purpose of including such an ad hoc ensemble is to obtain a rough sense of the gain in performance of the various systematic approaches to ensemble selection compared to that achievable using purely intuitive selection approaches.To facilitate a visual comparison of the various ensembles, the temperature points in the neighborhood of the heat capacity maximum for the various ensembles are graphically displayed in Fig. 4.
Figure 5 displays the calculated results for a typical equilibrium property, the heat capacity, obtained from parallel tempering simulations by using the various ensembles of Table I.Each simulation depicted involves 300 loops of 400 000 Monte Carlo moves per loop for each of the 21 members of the tempering ensemble.Each of these 400 000 moves consists, in turn, of one attempted exchange of configurations between adjacent, randomly chosen ensemble temperatures and a conventional Metropolis move for a single, randomly chosen spin in each of the remaining 19 ensemble members.
All tempering ensembles used in Fig. 5 produce accurate estimates of the Ising model's heat capacity.As indicated in Fig. 6, however, they do so with differing efficiency.In Fig. 6, we plot the standard deviations of the heat capacity estimates of the 2D Ising model obtained by using the various tempering ensembles with a fixed number of Monte Carlo points.The results shown in Fig. 6 are single standard deviation estimates obtained during the simulations corresponding to the results in Fig. 5. Clearly, the sampling errors vary depending on the choice of tempering ensemble.Owing to the similarity of the heat capacities upon which they are based, the performance of the CEI-S͑exact͒ and CEI-S͑20͒ ensembles are essentially indistinguishable.Interestingly, both of these CEI results are also effectively identical in performance to the ensemble produced by the optimization methods discussed in Ref. 16. Finally, we note that the sampling errors for the CEI-S͑4͒ ensemble, while somewhat less efficient than the CEI-S͑exact͒ or CEI-S͑20͒ results, are notably better than those generated by the ad hoc ensemble of Table I.This finding suggests that the CEI method can be of utility when used with even an approximate model of the heat capacity.
Although the standard deviation in a computed property of interest is perhaps the most direct measure of the quality of the underlying tempering ensemble, we present in Table II a second metric.We list there the likelihood that a configuration, starting at either of the ensemble's temperature extremes, successfully transits the entire ensemble via tempering exchanges without first returning to its starting temperature.As argued by Predescu et al., 10 this effective fraction is a useful metric for judging the quality of the sampling efficiency.The results in Table II are presented as an index N 0 , defined as the number of such starts required, on average, to produce a single, successful, direct transit.We see, for example, from Table II that, on average, roughly one in 2200 configurations "launched" from the T = 0.1 ͑or T = 10.0͒data stream via a tempering exchange successfully reaches the T = 10.0 ͑or T = 0.1͒ data stream directly ͑i.e., without first returning to the data stream corresponding to its initial temperature͒.We see that the ranking of the various ensembles when judged by this measure mirrors that found in Fig. 6 on the basis of standard deviations.Although not shown, we note that the ranking of the ensembles obtained in Fig. 6 and Table II agrees with that based on an eigenvalue analysis of the relevant transfer matrix that underlies the tempering exchange process.

IV. SUMMARY
In summary, we have introduced a intuitively appealing criterion for selecting parallel tempering ensembles.The essential feature of this approach is to require that increases in entropy between adjacent members of the tempering ensemble be constant across the entire ensemble.The resulting constant entropy increase or CEI approach is initiated with an estimate of the system's heat capacity obtained from experiment, preliminary simulation, or a suitable physical model.Applications to the 2D Ising problem indicate that this method is effective, simple to implement, and relatively robust with respect to its sensitivity to the quality of the underlying heat capacity model.
The focus of the present discussion is on techniques designed to deal with the application of stochastic quadrature to problems whose underlying probability distributions are sparse.Such problems present well-known sampling difficulties, difficulties for which parallel tempering methods have proved to be a valuable tool.Based on the results presented here, CEI techniques appear to offer a simple means for improving the efficiency of such tempering approaches.It is important to emphasize that caution must always be exercised when dealing with sparse distributions.As discussed elsewhere, 22 for example, tempering methods alone may be inadequate for the treatment of systems that exhibit lowtemperature, solid-solid transitions.The CEI method presented here is thus intended as a way to improve the efficiency of tempering approaches in situations where they are appropriate, as opposed to a means for extending their intrinsic applicability.I is a measure of likelihood that a configuration successfully transits the entire ensemble via tempering exchanges without first returning to its starting temperature.The relevant performance index N 0 is defined in greater details in the text.For comparison, the results obtained by using both a 21-member geometric and equally spaced tempering ensemble are also given.

Method
FIG. 1. Shown is the exact, analytical heat capacity for the 2D Ising model as a function of temperature.

FIG. 3 .
FIG. 3. Heat capacities for a 2D Ising model as a function of lattice size ͑L͒.The figure shows the analytical results for the infinite lattice ͑dashed line͒, for a 20ϫ 20 grid ͑solid circles͒, and for a 4 ϫ 4 grid ͑open circles͒.These heat capacity results in conjunction with the CEI method generate the parallel temperature ensembles listed in TableI.

FIG. 4 .
FIG. 4. A visual depiction of the various parallel tempering ensembles listed in detail in Table I.Each horizontal line of dots indicates the temperatures for that ensemble.Successive rows in the figure correspond to ͑starting from the bottom of the figure͒ the CEI-S͑exact͒, CEI-S͑20͒, CEI-S͑4͒, Ref. 16, and ad hoc ensembles of Table I, respectively.

TABLE I .
Shown are the temperatures of the various parallel tempering ensembles used for the 2D Ising problem.The results shown correspond to the CEI-S͑exact͒, CEI-S͑20͒, CEI-S͑4͒ ensembles, the ensemble of Katzgraber et al. ͓Ref.16͔, and the ad hoc ensemble described in the text.

TABLE II .
Shown for the various parallel tempering ensembles listed in Table