An assignment model is established for the targeted minimal divergence by introducing divergence indicators as a description of how well the edges of the paper scraps match. Then we solve the problem using the programming model. We design an algorithm for clustering by row - sorting within a row - sorting between the rows, intending a decrease of time complexity. At the same time the use of character cluster analysis and pattern recognition techniques helps reduce the error probability as well as human intervention in matching. The model and the algorithm are tested by experimental recovery of both English and Chinese double-faced paper scraps cut off either longitudinally or horizontally. The result indicates that the proposed recovery method is effective and accurate.