I have found a way to build new plugins for ImageJ; I don't know how well this will work with using other Java features at the same time though.
Using a little bit in ImageJ I found that doing certain processes in order does have a positive effect on possibly grabbing at some of the diatoms.
| |
| Original Image |
 |
| Image with background removed (Process in ImageJ). |
 |
| Image with background removed, converted to binary. |
 |
| Image with find edges process, converted to binary, with holes filled in. |
 |
| Image with background removed, converted to binary, and holes filled in. |
The images still select a lot of garbage, but they select the diatoms as well.
By performing a Gaussian blur and then reconverting to binary you get much more defined blobs.