Well there's good news and there's bad news....
The bad news first - storing inventory in an Amazon FBA location IS sufficient to generate a "nexus" in the state where that warehouse is located and, as a result it is sufficient to make you liable for sales tax on sales to customers who live in that State (unless you are in a category where that state offers an exemption...and so on.
...but you probably need to pay it. It's usually around 6-7% and often that includes the shipping too.
Here's a little vid I made on the subject.
The good news is that it's not really a big problem. Even as an FBA seller with 12 months selling history and several inventory locations around the USA it is unlikely that you will have a significant liability. It only affects sales in the affected state so the big ones are going to be California and New York where the populations are high. States like Indiana and Arizona where lots or the FBA warehouses are located may not contribute all that much to your revenue anyway - and even then it's only 6-7% cost.
So what about the admin hassle? Well, you can either register with Amazon upfront and then contact each state where your inventory is stored to register with them too or you can use a service like TaxJar - you still have to register with the individual states but TaxJar automates a lot of the process for you, gives you an indication of your taxable sales right out of Amazon (and other platforms like Shopify) and provides guides on how to do it all with links to the places where you register. Some of the states are clearly not geared up very well for international sellers yet - kind of demonstrating how confusing this all is for everyone. What I mean by that is they don't let you fill out your address properly (you have to pick a US state) and they insist on a Social Security Number (SSN) in the online form - which you can't get as a non-US resident. SO you have to do it by mail. That is just one example.
In the end, Sales Tax is best automated as far as possible. There is some set-up pain but it's quite a low impost and you don't ever want to get on the wrong side of the tax authorities.
If you are interested in automating, this affiliate LINK will get you started with TaxJar. Please note if you use this link and sign up with Tax Jar, I may get a discount on my own TaxJar fees - because I use them too!
If you want to collect Sales Tax right in your Amazon Seller Central Account (get Amazon to do it for you) then here's a TaxJar guide.