Seven Briefs on Semantic Communication and 6G
As the research on 6G wireless systems is getting traction, one of the phrases/technologies that is creating a lot of buzz is semantic communication. For example, there is currently an special issue dedicated to semantic communication in IEEE JSAC, as well as a number of dedicated workshops and special sessions at conferences. Semantic communication is seen as one of the technologies that may have a transformative potential in 6G, along with integrated communication and sensing, intelligent electromagnetic surfaces, the unavoidable Artificial Intelligence/Machine Learning and several others.
As it is common, this trend creates two polarized points within the research community, the first one leaning towards "there is nothing new here, I did it 20 years ago"; the second leaning towards "everything is new here, we have to completely change the way we did the things before". It is thus advisable to take the Confucian approach of the middle way and use a cautious optimism in examining the premises and promises of semantic communications, in particular with respect to its impact on communication engineering/theory.
The seven briefs below are reflections that are based, to a large extent, on my participation in recent workshops on the topic, most notably the one at EuCNC that featured excellent talks from my felllow researchers, as well as a vibrant panel session.
- What does "Beyond Shannon" mean?
The way "Beyond Shannon" is commonly motivated is to look in to the three types of communication problems, as defined by W. Weaver in the introductory note to Shannon's paper. For completeness, we reproduce them here:
- Level A: How accurately can the symbols of communication be transmitted? (The technical problem.)
- Level B: How precisely do the transmitted symbols convey the desired meaning? (The semantic problem.)
- Level C: How effectively does the received meaning affect con- duct in the desired way? (The effectiveness problem.)
The observation is that the traditional domain of information-communciation theory is in the Level A problems, while the problems at level B and C are not treated. Consequently, going "Beyond Shannon" means to consider the problems at Level B and Level C, dealing with semantic and effectiveness. While I do not disagree with this viewpoint, as back in 2019 we have advocated that at the first 6G summit, it should be noted that the phrase "Beyond Shannon" can be unpacked into multiple aspects.
The long version of "Beyond Shannon" is "Communication models that reach beyond the mathematical model defined by C. E. Shannon in 1948, as well as its derivatives." In order to revisit this powerful mathematical model , we have to try to change the way we are looking into its tacit, underlying assumptions. In the sequel I am referring to the basic model of Shannon, in which a single transmitter communicates one-way to a single receiver, see Figure 1. There are many other multi-user models that span out of this model, but for the sake of this discussion, the basic one-way model is sufficient.
Shannon's model is the one of push communication, in which: (1) the transmitter selects the message randomly; (2) the receiver is always interested to hear what the transmitter has to say; (3) the objective is to avoid error in the transmission, while, strictly speaking, the communication procedure (or the channel coding) does not care what happens upon error. To go beyond the model of Shannon could be, for example, what happens if the transmitter is not selecting the message randomly, but rather based on a query from the receiver? Or selecting the message in order to convey some high-level concept to the receiver (semantics) or some actuation command that need to achieve a certain goal (effectiveness)?
A final note on this part is that, the division of communication problems into technical, semantic and effectiveness, while intuitively appealing, may be limiting. For example, in linguistics one talks about syntax, semantics, and pragmatics, which is basis for a different perspective on the Beyond-Shannon problem.
2. Relation to joint source-channel coding
One of the aspects of "Beyond Shannon", exemplified above, is that the channel coding, or the pure communication, does not care what happens if an error occurs. This is a question that should be handled by the higher layers. In fact, what happens after an error occurs is not beyond the model of Shannon, as it can be handled by the source decoding; Figure 1 provides a layered perspective on the Shannon model, where the source coding is the higher layer. Source coding is the operation of compression, which takes as an input arbitrary source and the objective is to output a sequence of uniformly distributed random bits, which cannot be compressed fuether. This sequence of random bits is used to generate uniformly distributed messages that appear as inputs to the lower layer, the channel coding. Looking jointly into the source and channel coding enables us to see how much the information decoded by the receiver is different from what the source at the transmitter wanted to communicate. Setting up an appropriate distortion function can serve as a measure for a precision, or, in other words, how much the reconstructed signal deviates from the desired message. However, this has more general consequences, since deviation can be defined for an abstract space, e.g. within a space of concepts, such that we can measure how much the meaning of a decoded/reconstructed concept deviates from the intended meaning. Isn't that also a semantic communication, just within appropriately defined set of sources and messages?
As a bottomline, joint source-channel coding is a very powerful paradigm for communication and, every time we think it cannot be applied to some problem, it is better to think more, rather than dismissing it.
3. Layers of meaning in a communication protocol
Let us take the hypothetical case in which we put an antenna in the air that detects a single wireless data symbol. What is the meaning of that symbol? In fact, looking at a single symbol we cannot even detect to which type of modulation constellation it belongs, whether it is BPSK or 16-QAM. In principle, we can also try to detect a whole sequence of symbols and infer what kind of modulation is being used, without this explicitly being signaled before. Yet, in a given practical communication system the way to arrive to a correct interpretation of a given symbol is to look into the prior knowledge obtained through channel estimation or through some prior signaling that explains how the symbols should be interpreted in terms of modulation and coding. In general, the meaning of wireless data symbol is determined by a control information or metadata that is sent in relation to that symbol. For example, the data in a packet sent at the link layer can be interpreted correctly by decoding the control information, such as packet source, sequence number, etc. Following this line of reasoning, the interpretation of a metadata requires transmission of metametadata, and furter metametametadata, etc. This has to stop somewhere; the stopping point is the common, pre-agreed state of the communication protocol, with default parameters and codewords.
A possible interpretation of the layered protocol paradigm, the meaning of symbols/data transmitted within a given layer is interpreted by the higher layer. There are thus multiple layers of meaning within a communication protocol, starting from the lowest layer ("What is the meaning of this noisy symbol?") and climbing up to the highest, application layer ("What is the meaning of the message X in the context of a knowledge graph Y?").
Two things follow from this discussion on layered communication:
(1) Semantics of a communication can be defined in a different sense, starting from the narrowest sense at the lowest layers to the widest sense at the application layer;
(2) Control information/metadata is tightly related to the semantics of the communication as it tells the corresponding module of the communication protocol how to interpret the data. This indicates that semantic communication may have an impact on the design and exchange of control information.
4. Compression and non-probabilistic information
In the context of wireless communication channels that have a highly variable quality, semantic communication is seen as a way to use those channels to convey the intended meaning rather than only ensuring that the message is transmitted correctly. In a similar spirit, goal- or task-oriented communication aims to have communication that is sufficiently good with respect to the given goal or task, thus promoting the effectiveness of communication. In both cases, whether it is semantics or effectiveness, we have a situation in which information should be suitable compressed to be able to pass through the channel with an acceptable quality. Here acceptable quality means that the meaning/goal of the transmitted message is conveyed/achieved, with high probability, even if the channel introduces errors. Yet, when talking about semantic or goal-oriented communication, there is some reservation to identify it with a form of compression is it seems that "there is more to it" (recall the above discussion on joint source-channel coding). In fact, compression can be seen as a form of learning. For instance, the celebrated Lempel-Ziv-Welch algorithm learns the patterns within the data in order to be able to make a succinct, compressed version. A formula that describes a physical law is a compressed version that embeds all the possible manifestations of that law.
A valid question is, though, whether the compression should be probabilistic or based on other criteria. Shannon's notion of information is probabilistic, as it measures the entropy, that is, the level of surprise and uncertainty contained in some phenomenon. In that sense, it is difficult to see how is information generated by a proof of a mathematical theorem, since each step of a proof follows deterministically from the previous step. The information contained in the proof is a logical, rather than probabilistic phenomenon. Let us then consider communication between Alice and Bob, in which Alice wants to communicate to Bob a proof of a theorem. What is the most efficient way to communicate it? Or in other words, how to compress the proof so that Bob gets it with minimal number of uses of the communication channel? Proof of a theorem, as limited as it may be in terms of representing knowledge, could be a good model for studying semantic communication and transmission of non-probabilistic information.
5. Semantic communication and the increasing intelligence in the network
In common communication-theoretic models, the communicating parties are tacitly assumed to have static capabilities, such that the codebooks for communication of data and metadata, as well as the protocol states, are predefined and used throughout the system lifetime in an adaptive way. This is somehow in contrast to the trend by which the intelligence in the network nodes and end devices grows over time. The higher the intelligence of the communicating nodes and the more the nodes know about each other, the more they are in the position to make customized communication protocols, inventing along the way methods for more effective communication. As indicated above, control information is tightly related to the meaning and interpretation of given communication. Wireless nodes that are interacting over a longer time can try to invent their signaling and data representation to convey meaning in a way that corresponds to the increased intelligence in the nodes. For example, systems that share the same spectrum over a longer period can create a set of messages to coordinate each other and this set of messages can be adapted to the environment/context where the nodes operate. An important aspect of semantic communication is how to take into account these long-term changes in the environment and intelligence and lead to transmission techniques and protocols that can adapt to it, rather than relying on manual design of control information in the system.
6. Semantic communication and 6G
The brain is an entity stored in a dark box, but is capable to create most vivid sensory experiences. It is connected to the outer world through multiple sensory communication channels coming from different stimuli, such as audio or visual, orchestrate and combine them in purposeful way and play them out for us as a synchronous now moment.
I am making this prelude as several visions towards 6G point out to a synergy between communication, sensing, positioning and, ultimately, everything connected through intelligence. Following the "triangle story" about 5G (eMBB, mMTC, and URLLC), 6G can be seen as an augmented triangle, as in Figure 2 below.
In this context, semantic communication gets a broader context, encompassing the sensing and positioning/localization/orientation. Sensing can be seen as a communication in which we are not in control of the information source, while positioning can be seen as a type of metadata to communication, in addition to the fact that it is a data on its own right. In a similar way in which the brain orchestrates multi-channel inputs, semantic and goal-oriented communication towards 6G can be seen as a fusion of various communication modalities. This brings us again to the issue of joint source-channel coding knowing that we have other sources of information (sensing, positioning) as well as the design of control information that helps to fuse information coming from different modalities.
7. The survival of Layering
Every time some groundbreaking novelty in communications is at the horizon, there is a growing revisionist feeling towards the layered paradigm. Twenty years ago there was a lot of discussion on cross-layer design and how it would change everything; with the topic of semantic communication, this revisionist feeling is also awoken. Indeed, joint source-channel coding is, by definition, a cross-layered paradigm. The separation of source and channel coding is a layered design, where the interface between the two layers is a sequence of random bits. While there are cases in which separation is optimal, there are models in network information theory where the separation of source and channel coding is clearly suboptimal. In practical systems there seems to be always something to gain, in terms of performance, by tightly coupling the operations of layers.
Semantic communication also spans multiple layers and inevitably leads to the question — should we abandon layering to gain performance? One thing to keep in mind is that layering and modularized design has an architectural advantage, as it offers ways to scale the system and address unforeseen use cases and scenarios. Take the Internet as an example, which is largely an architectural success, offering capability to send packets through a network and enabling build up of plethora of services. This is at a price of some performance loss compared to the systems that are dedicated (cross-layered) to a particular type of service; for example, an analog telephone is a fully cross-layered system. A possible way in which semantics could work with the layered paradigm is by introduction of semantic plane that spans across different layers.
Whatever the further developments of communication technology will be, some form of layering and modularization is poised to stay around, potentially transformed by the increased intelligence in the nodes. Winston Churchill once said : “Democracy is the worst form of government — except for all the others that have been tried.”; we can probably make the same statement about layering in communication systems.
