This became interesting and I googled it and found interesting document. So using browser cookies is actually a feature and was intentionally done and worked in Java Web Start and Java Plug-in applications.
For my case it was totally undesirable and I wanted to turn it off. Fortunately, it can be done quite easily (though more complicated then it should be):
CookieManager cookieManager = new CookieManager(); CookieHandler.setDefault(cookieManager); cookieManager.setCookiePolicy( CookiePolicy.ACCEPT_NONE);