There isn't a preview button per se but there are two ways to see the actual data, rather than the mail merge fields, in a mail merge.
Here's a PDF from my workbooks with two approaches. One is just printing several or all of the records to one OpenOffice Writer file, then opening that up to look. The other is to click a button in the database view area, which lets you see just one page of records (one, for a mail merge letter or an envelope, or a whole label sheet full for labels).
The PDF starts out with standard mail merge printing instructions, and the two preview techniques are at the end.