Moojun Kim  Beomcheol Kim  Jindae Kim

10.5626/JOK.2023.50.2.150

source code differencing technique edit script change mining Empirical Study

This study investigated the difference between descriptions of code changes made by source code differencing tools and humans. We applied two popular source code differencing techniques to collected code changes. We found that these tools often generated different descriptions for the same changes, and only 3% of the changes have the same descriptions from both tools. On the other hand, human participants agree on change descriptions for 50% of the given changes. Furthermore, many of the different descriptions were caused by simple mistakes. If we ignore differences caused by these mistakes, human participants described 71% of the changes similarly. We also compared change type and entity type of edit scripts generated by human and the source code differencing techniques for the same changes. We found that the techniques generated the same description as humans for only 8.20～35.65% of the changes, which indicates that these techniques require significant improvement to provide descriptions similar to human’s.