You already know this ... when you sell on Amazon, those aren't your buyers. Yes, they're buying your merchandise. But they won't ever be anywhere near as loyal as the customer who goes to your website to purchase something.
Now, if you can make the numbers work and the transaction is highly profitable? Sure, have at it!
But go in knowing that you are to Amazon as Coach is to Nordstrom.