A joint fermi-gbm and ligo/virgo analysis of compact binary mergers from the first and second gravitational-wave observing runs