Back in the days when such statements were not considered racist, people in movies, television, and various other media used to say that the Japanese were "inscrutable". I'm not sure exactly what aspect of Japanese behavior elicited this widely held observation, but the statement is supposed to reflect the idea that the Japanese are difficult to analyze and understand. One thing I can say for certain is that it does not have anything to do with language. Plenty of people learn to read, speak, and write Japanese language. Another thing I can say is that the idea that Japanese people are incomprehensible to outsiders is one that the Japanese themselves love to perpetuate.
Part of the reason I believe that the Japanese believe that outsiders can't understand their culture's depth and breadth is that they lack perspective on outside cultures. They don't know how their culture differs because they don't have intimate knowledge of other cultures as a point of reference. They can't explain the differences because they don't know what is different except in relatively isolated cases. Few people in either culture have a macro view of the mechanics of the cultures they're trying to understand so we are left with a lot of micro views of things like deference to authority, respect for the aged, notions of homogeneity and conformity, etc.
I'm sure that somewhere someone has written books explaining the things I'm about to say and that someone in an academic ivory tower has made it their life's work to understand and explain such things. I'm not attempting to compete with such authorities or to say I know what they know, but there are some things I've realized which have provided me with a different perspective and a better understanding of a particular aspect of Japanese culture which tends to drive many foreign people who live here mad. That is the way in which the Japanese handle matters of rule and law on a "case by case" basis. That is, they have rules which they will recite chapter and verse, but then they will bend those rules on a whim.
For foreign folks, the "case by case" culture is maddening because we never know where we stand and we always expect to be standing in the line which says, "you're going to get screwed over." The latter is a bit paranoid, and probably fueled by errors we make when dealing with Japanese business and bureaucracy, but I'm sure that prejudice is applied at times as well.
Just to be clear, I will offer an example of a typical "case by case" situation which affects our lives as foreign folks. This is the immigration situation and the requirements for visas and visa renewal. When you apply for a visa, you get an application form and a list of documentation that you must submit. In the vast majority of cases, you hand over the documents and you get your stamp and move along. However, as part of the information you receive, there is essentially a statement that the immigration office can demand any other documentation of you that they wish and can refuse you for any reason whatsoever. They don't have to point to any rule to demand that you offer up your mother's birth certificate, your foreign bank account's address, or your grandmother's marriage license. If they want these things, they have every right to request them because they have the power to handle every situation on a "case by case" basis. There is no promise that you will be treated equally to other applicants or that the same demands will be made of each applicant. Now, I don't think that such demands are made, though I wouldn't be the least bit surprised if certain people were refused renewals based on this flexibility and power.
The interesting thing about the "case by case" culture is that it can benefit as well as take away from you in a given situation. People who have power to make life harder for you also have the power to make it easier. They can waive punishment if you apologize or they can make your case an exemption when the other 1000 cases before you had to do exactly the same thing. This does happen in Japan and I think it'd probably happen more often for foreign folks if they knew how to show regret and accept responsibility in exactly the same manner as Japanese people do.
Unfortunately, we foreign folks, by and large, are accustomed to rules that are hard and fast. We like our rules this way because they guarantee that each person is treated exactly the same and that we are all equals. Without such rules, there is the possibility of prejudice and abuse of power. We like to have everything spelled out for us so that we can operate under the expectation that we can be guaranteed a certain result when we cross all of the "t's" and dot all of the "i's". Under such a system, we have the power to control the outcome because we know exactly what to do and we know that no one in an authority position has the power to deny us if we do everything we're supposed to.
If you compare the usual Western structure to the Japanese one, you'll notice that the Japanese situation places a great deal more power in the hands of the authorities making the decisions. You are, essentially, at the mercy of their whims should they choose to exercise them and you are powerless to do anything about it by tapping on a rule book or quoting a law because they have a trapdoor in most cases which allows them to say 'and whatever else we want from you or want to do to you.'
Fortunately, most of the time the people who have the power do not abuse it nor do they use their power to make your life difficult. Generally speaking, everyone just does what they're supposed to do and all goes smoothly. Mainly, in my experience, knowing about the way the case by case culture works helps when things go wrong. If you approach the person in authority in a manner which does not challenge their power and places you firmly as someone who is of inferior status, highly apologetic, and sincerely regretful that something has gone awry, that flexibility is used to help you rather than harm you.
Unfortunately, most expatriates don't approach dealing with various authorities and businesses in Japan in this fashion. Because we are accustomed to knowing exactly what our rights are and what sort of power we have, we get aggressive and insistent about what is supposed to happen and what is "right". In other words, we challenge the person in authority's right to make various requests and choices which only serves to make them all the less likely to want to help us or let us off the hook.
To us, following the Japanese way is degrading. It makes us feel like we're allowing ourselves to be cowed into submission or lowering ourselves in order to cooperate. In American culture in particular, the whole idea of figuratively kneeling before an authority in submission or putting yourself at their mercy is repugnant, cowardly, and a surrender of your independence and rights. In the end though, it's not about your ego. It's about a culture that is comfortable with having two faces, the true face (honne) and the public face (tatamae). The Japanese don't feel degraded by having to use a different face in order to make things operate smoothly in most cases. Their core ego structure is not centered around the public face. It's just something they put on when needed and take off when unnecessary. They oil the wheels when needed to make things go smoothly and move on to other things.
In the end, we can only be true to who we are. As an American, I don't know that I could ever be entirely comfortable with the "case by case" nature of Japanese life. However, one thing I've realized is that this system is not in place as an excuse to be prejudicial. It's about power, not prejudice, and keeping it in the hands of one group of people rather than in the hands of the average person. It's something that has developed as a result of Japan's history just as the egalitarian nature of America's laws has developed because of its history. If I want to continue living here, it serves me far better to accept that rather than to fight it.