Can I construct a custom Qml item that contains a placeholder child?

Have you tried making your item the default property (http://qt-project.org/doc/qt-5/qtqml-syntax-objectattributes.html#default-properties) of MaskedItem? Alternatively you could try working with a Loader element.