weixin_39726379
2020-11-27 13:38 阅读 316

org.bouncycastle.util.encoders.DecoderException: unable to decode base64 string: invalid characters encountered in base64 data

Hello All,

Could you please help me figure out the cause of the exception? In my project I've updated version of Bouncy Castle: from bcprov-jdk16-146.jar to bcprov-jdk15on-1.57.jar And now in my integration test I get an exception: org.bouncycastle.util.encoders.DecoderException: unable to decode base64 string: invalid characters encountered in base64 data at org.bouncycastle.util.encoders.Base64.decode(Unknown Source) at com.project.isx.server.AuthenticationTicket.<init>(AuthenticationTicket.java:570) at com.project.isx.server.AuthenticationTicket.readTicket(AuthenticationTicket.java:388) at com.project.isx.server.ResourceProcessor.init(ResourceProcessor.java:218) at com.project.isx.server.Processor.createResourceProcessor(Processor.java:756) at com.project.isx.server.Processor.process(Processor.java:622) at com.project.isx.server.AccessGateway.service(AccessGateway.java:134) at com.project.test.isx.server.resource.enrollment.TestNGPinSelfService.send(TestNGPinSelfService.java:174) at com.project.test.isx.server.resource.enrollment.TestNGPinSelfService._testPinSelfService(TestNGPinSelfService.java:148) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at org.testng.TestNG.run(TestNG.java:1064) at com.project.runtests.TestNGContainerSuite.run(TestNGContainerSuite.java:379) Caused by: java.io.IOException: invalid characters encountered in base64 data at org.bouncycastle.util.encoders.Base64Encoder.decode(Unknown Source) ... 31 more

Maybe I need to update the another .jar because of the BC has some dependencies? Thanks!

Best regards, Iryna

该提问来源于开源项目:bcgit/bc-java

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享

26条回答 默认 最新

  • weixin_39954464 weixin_39954464 2020-11-27 13:38

    The exception is thrown because the data is not valid Base64 encoded. You should check the data to see why it is not correctly base64 encoded.

    点赞 评论 复制链接分享
  • weixin_39726379 weixin_39726379 2020-11-27 13:38

    Thanks for the replay. Could you please give me advise how can I do this? Do I need to verify that data before method calling? Thanks!

    -Iryna

    点赞 评论 复制链接分享
  • weixin_39840635 weixin_39840635 2020-11-27 13:38

    Yes, you have to make sure your data is valid base64 before calling Base64.decode().

    Maybe take a look at the base64 string being passed to decode, and see if it contains any characters other than those contained in this table or the padding character =.

    点赞 评论 复制链接分享
  • weixin_39726379 weixin_39726379 2020-11-27 13:38

    My tests run on Jenkins and in certain cases I get DecoderException. the encoded string: rO0ABXQAAzEuMnQAJDgyYjUyY2YxLTRmYjctNDkyZC1iNTJjLWYxNGZiN2I5MmQ1MnQAJGNmNGVjYTY1LTZkYTItNDMzNS04ZWNhLTY1NmRhMjUzMzU2MXNyAA5qYXZhLnV0aWwuRGF0ZWhqgQFLWXQZAwAAeHB3CAAAAV70VY8+eHNyABFqYXZhLmxhbmcuQm9vbGVhbs0gcoDVnPruAgABWgAFdmFsdWV4cAB0AAB3BAAAAAF0AANQV0R6AAABKgAAASYwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC9Cqkw7uZ2ScW4GHcgEXGfx3Jj/ssUhhT7UN6QyztzCZPVB4nuTyL8cFrIw56VU1g/BDBkE31HYOXRYJKH0oMGg7qCJV4ucb81uCXcyLmm6IVX8xEY3Ku4hRKyrvk6xEWzmlnUMG2JHjrikpD8dd45/YuSzUpJ8vfhcHil9PM/MtvhEHNAQTf6wbM+vWQ3p69OMi5y8UidLl1FvBsIrGWpN8AnVEiOhGcEEQO5bzCFkFx0gTilW6A6/85XmB7m5Jvt6uCvUBVruTsZlq0AIDYBLukodsDzZFgpvUnZ5GL+lH72VKr1T54CFi0yWNWSGgR7CD6Jtg6Dotk9D+KuZJqhAgMBAAF3iAAAAesAAACAot1b9EojVTgdVWlSMsj89fDAH8hL+r3zRuYTd44ul5BwVXGDKyszRNmdu8WVRp/0uHdpNIvnueo3tLh8R2JU9Byab6Eje21ej1z54OA9E1iC3s6yvlMTrurVk6ZBPDyuCDRFwcfvCneJa+AqIwXHl05mRFp4WRIv1K0BPbOv+s0= But if I put this string into my local dev environment, everything is ok. Also I used regexp and this string is matched with [A-Za-z0-9+/=]. Could you please take a look? Thanks a lot!

    点赞 评论 复制链接分享
  • weixin_39840635 weixin_39840635 2020-11-27 13:38

    Hmm, that encoded string seems fine to me too. Could something in your code be adding \ as an escape character before /? (That's an issue I've had in the past.)

    点赞 评论 复制链接分享
  • weixin_39726379 weixin_39726379 2020-11-27 13:38

    I thought about this too... It is surprising that this is the same code with the same data. (Jenkins and locally). And this test is failure from time to time. Not always. Looking at the log, I don't see anything special. I reproduce all the same locally, but I don't get an exception.

    Thanks!

    点赞 评论 复制链接分享
  • weixin_39726379 weixin_39726379 2020-11-27 13:38

    I've read release notes for 1.57 version of Bouncy Castle. And than mentioned about: The Base64 encoder now explicitly validates 2 character padding as being "==". Maybe is it the reason? Because before this we use earlier version and all test ware passed. It's my thoughts...

    点赞 评论 复制链接分享
  • weixin_39726379 weixin_39726379 2020-11-27 13:38

    Could you please take a look on this issue? Thanks a lot!

    点赞 评论 复制链接分享
  • weixin_39840635 weixin_39840635 2020-11-27 13:38

    Sorry, I'm out of ideas! You may have to wait for an explanation from an actual developer on the project.

    点赞 评论 复制链接分享
  • weixin_39726379 weixin_39726379 2020-11-27 13:38

    Thanks a lot for the replay! Could you please tell me who can help me? :)

    点赞 评论 复制链接分享
  • weixin_39840635 weixin_39840635 2020-11-27 13:38

    Have you tried asking on StackOverflow?

    点赞 评论 复制链接分享
  • weixin_39619451 weixin_39619451 2020-11-27 13:38

    I don't think the exception is being caused by what is the case - the supplied string is clearly correct. The only thing I can think of is that a decode method is being called with mutable data and some other thread is overwriting it.

    点赞 评论 复制链接分享
  • weixin_39726379 weixin_39726379 2020-11-27 13:38

    I would like to add that my code has not changed for a long time. This problem started when I updated the library version to 1.57. Thanks!

    点赞 评论 复制链接分享
  • weixin_39619451 weixin_39619451 2020-11-27 13:38

    Was there anything else you updated? I think I've worked out what's going on though.

    There is a change between 1.46 and 1.57 - the check for invalid input was approved.

    I would suggest grabbing a copy of org.bouncycastle.util.encoders and creating your own package for it. Use that in the test, but modify the code so that where you see "invalid characters encountered in base64 data" you print the hex value of the characters concerned (with & 0xff). My guess is you'll find that occasionally a control character is coming through - previously this would have evaluated to zero, now it's flagged as an error (as it should be). If the code has been otherwise working, you'll probably find it at the end of the data string.

    点赞 评论 复制链接分享
  • weixin_39726379 weixin_39726379 2020-11-27 13:38

    Thanks for the advice! But when I've added the classes locally, I got the exception: SecurityException: class "org.bouncycastle.util.encoders.Base64Encoder"'s signer information does not match signer information of other classes in the same package

    Could you please help me? Thanks!

    点赞 评论 复制链接分享
  • weixin_39619451 weixin_39619451 2020-11-27 13:38

    You'll need to change the package name - call it something like test.bouncycastle.util.encoders

    点赞 评论 复制链接分享
  • weixin_39726379 weixin_39726379 2020-11-27 13:38

    Thanks for the reply! I’ve tried this and added additional logging in decode method (output encoded string at the beginning and output hex value when throwing the exception). And I’ve seen that before call Base64.decode, we call URLDecoder.encode (from java.net). In this method all + symbol is replaced to spaces. So.. We always call Base64.decode with string which contains spaces. But for format for Base64 data is [A-Za-z0-9+/=]. Always. And locally this test is passed always. On Jenkins is intermitted. But this method throws the exception intermittently. Could you please comment on this? Thanks!

    点赞 评论 复制链接分享
  • weixin_39619451 weixin_39619451 2020-11-27 13:38

    Okay, the Base64.decode() will ignore the spaces, but by ignore I mean they'll be removed from the stream. If these are replacing spaces this will invalidate the encoding if the padding doesn't line up with the new message length (so if there are 3 '+' the 3 ' ' will produce a valid base64 string, but it won't decode to the correct data, on the other hand if there are 1, or 2 '+' it will invalidate the padding and the message will fail). Previously the ' ' would have just meant a junk decoding, but no error, would have been returned.

    Does that sound like it explains what you are seeing?

    点赞 评论 复制链接分享
  • weixin_39726379 weixin_39726379 2020-11-27 13:38

    Thanks for the reply. If I call this method (which has my additional logging): if((var6 | var7 | var8) < 0) { SystemLog.logMessage("var4 != this.padding && var5 == this.padding"); SystemLog.logMessage("var6: " + var6 + " " + (var6 & 0xff) + ", var6 < 0 " + (var6 < 0)); SystemLog.logMessage("var7: " + var7 + " " + (var7 & 0xff) + ", var7 < 0 " + (var7 < 0)); SystemLog.logMessage("var8: " + var8 + " " + (var8 & 0xff) + ", var8 < 0 " + (var8 < 0)); throw new IOException("invalid characters encountered at end of base64 data"); } with encoded string which contains spaces, I get the next log: var4 != this.padding && var5 == this.padding. 2017-10-25 11:53:15,492| sso-win INFO [TestNGContainerSuite]: var6: 11 11, var6 < 0 false. 2017-10-25 11:53:15,492| sso-win INFO [TestNGContainerSuite]: var7: -1 255, var7 < 0 true. 2017-10-25 11:53:15,492| sso-win INFO [TestNGContainerSuite]: var8: 28 28, var8 < 0 false.

    Coupld you please comment on this?

    点赞 评论 复制链接分享
  • weixin_39726379 weixin_39726379 2020-11-27 13:38

    So.. Variable var7 equals -1 and its hex value equals 255: var7 = -1; hex value = 255, var7 < 0 Do you have any thoughts? Thanks!

    点赞 评论 复制链接分享
  • weixin_39726379 weixin_39726379 2020-11-27 13:38

    For example, if I call Base64.decode("AZaz+/="), everything is ok. But if I call Base64.decode("AZaz /="), I get the DecoderException. " " instead of "+".

    点赞 评论 复制链接分享
  • weixin_39619451 weixin_39619451 2020-11-27 13:38

    That's correct - the second base64 is invalid, there should be 2 '=' as ' ' is ignored.

    点赞 评论 复制链接分享
  • weixin_39619451 weixin_39619451 2020-11-27 13:38

    Mystery appears solved.

    点赞 评论 复制链接分享
  • weixin_39726379 weixin_39726379 2020-11-27 13:38

    With Base64.decode everything is ok :) In our test we need use URLEncoder.encode before call URLDecoder.decode. So this was a flaw in the test, which the more strict implementation of BouncyCastle detected. Thanks a lot for your helping , , .

    点赞 评论 复制链接分享
  • weixin_39612817 weixin_39612817 2020-11-27 13:38

    Looks like there is UrlBase64 encoder as well, but it produces non-standard base64, so the tradeoff - no need to do additional URL encoding/decoding, but you must use the same class to decode.

    点赞 评论 复制链接分享
  • weixin_39726379 weixin_39726379 2020-11-27 13:38

    Since this is a test, we simulate a behavior of one system. Therefore, in this case, first there is an encoding and then decoding.

    点赞 评论 复制链接分享

相关推荐